Hi, Review Board founder here. Let me address these real quick.
1) This has come up before. We'll probably add it. Most places that ask for it really are wanting something like an auth backend that ties into another service, like LDAP/ActiveDirectory (which are supported natively), a Single Sign-On service (which we'll be adding support for in Power Pack, our add-on product), or something more custom. For those who do want to add via the API, it's something I imagine we'll do in the next major release or two. I'll make sure it gets on the roadmap.
2) Review Board 4.0 (which is in development, alongside the upcoming 3.0) features DVCS support that will make it a lot easier to update patchsets and to keep them together as a whole without having to squash commits or create a review request per commit or anything else like that. Reviewers can view each commit individually, or the whole thing squashed, or a range squashed, and we'll help visualize how the history of those commits evolve with each update (such as adding new commits to HEAD, squashing commits together, etc.). It'll be really nice when it's done.
We're also looking at adding push-to-commit through our in-development Git (and eventually Mercurial) wrapper service, rb-gateway, so that you can push a branch there and have it appear on Review Board, and keep updating that branch to update the review request.
And, we're prototyping support for pull requests (first for GitHub, and then for others later), so that one can submit a pull request and have it basically appear in Review Board. This is still in the very early stages, but it's a priority. May not be relevant to your workflow, but thought I'd mention it, because that's asked a lot.
Again, Review Board 4.0 is when a lot of this will make an appearance, but I'll try to get account creation via the API to happen in 3.0. (Just added it to our roadmap.)
We use this in the development of Juju. We also have a bot that creates Review Board reviews from GitHub PRs, and updates the diffs when the PR is updated.
Sounds cool :) I'd love to hear more about your setup if you'd be up to talk about it further (christian@beanbaginc.com).
Native support for OAuth2 is planned to land in 3.0 as well. We're focusing a lot on improvements for third-party integrations and enhanced extensibility with this release (along with some cool new review improvements).
Oh I should be clear (misunderstood that extension). We're adding support for using Review Board as an OAuth2 provider, not to use another service for auth, so that services can better integrate into your server without having to share credentials.
My usecase for item 1 is a bit special, I can admit. The pain point came up when I tried to set up RB for testing, and interestingly enough both items tie directly into each other.
- Reviews need to be generated automatically when suitable trigger conditions are met. This means that there has to be a dedicated user account in RB that can create reviews on behalf of other users. Reviews also need to be automatically updated when new changes land in the branch under review.
- While setting RB up, it's not at all uncommon to do a round of install/provision/repopulate iterations. Without the ability to generate users and assign their permissions through an API the automation and iteration go out of the window.
- The account permissions for RB are very granular. I don't want to manage these permissions via a web-UI, I want them populated and updated based on reviewable commits. Hence, everything must go through config management. Yet another web-UI for managing yet another set of accounts and permissions is ... increasingly repulsive.
I can understand where the permission and account creation friction comes from. For you, RB is a product. (Hell, it certainly is one!) For me, RB is simply a tool, and as such it has to integrate with other tools. Code review tool is an extremely important one, but without fluid integration with other development workflow tools it becomes an additional burden to maintain. For developers who don't need to worry about integration efforts, yet another extra account adds to the cognitive overhead.
Furthermore, I realise your permission model is tightly coupled with the underlying architecture, which itself builds on top of Django. Exposing the kind of functionality I ask for requires to expose and nest quite a few internal API layers. I do genuinely see why that has not been roadmap item so far. Codewise that is a big undertaking.
As for item number 2... you just made me drool. I haven't seen a tool yet that would allow to easily see change comparisons with dynamic interdiff windows. Having visibility into how a change set has evolved, and what the compound intermediate results have been between any given steps would be a huge boon. (No, `git diff ref1..ref2` is not ideal. On the other hand something like `git show ref1 ref2 ref3..ref6 ref7..ref8` gets a lot closer.)
Once you have native DVCS support and push-to-commit for fluid integration between other tools, RB has the ability become "just another tool". I see that as a good thing :)
I haven't yet tried phabricator (evaluating it is on our infrastructure team's roadmap), but so far it looks like it tries to do too much.
You guessed correctly, the PR integration is not relevant for me, but I can definitely see how it would better integrate with lots of teams' workflows. No wonder it gets asked for.
> Again, Review Board 4.0 is when a lot of this will make an appearance, but I'll try to get account creation via the API to happen in 3.0. (Just added it to our roadmap.)
Thank you. Thank you so much.
NB. I am a big believer in tooling and workflow. Related to that, I have developed an increasing intolerance to anything that cannot be configured via config management. Changes to infrastructure are no different to changes in code - everything needs to be reviewable and versioned. (Which also means that the infrastructure must be rebuildable with minimal manual steps.)
Hey Phabricator dev here, would love to see you take a better look at Phabricator as well. I'm not sure that we really "do too much" as our product is guided mostly by the tenants of code quality and developer efficiency. Specifically we feel it better to have these tools all integrated so developers spend more time just building features. Some basic overview for code review would be:
> would love to see you take a better look at Phabricator as well.
I will. Phabricator was brought into my attention some time ago, not surprisingly around the same time I was looking at the alternatives. I think I should expand on the "does too much" part a bit, because in using that expression I clearly managed to upset you.
Let me state one thing upfront - good engineering workflow is critical. That is also a reason why introducing new tools to existing workflows is so damn hard. And that is precisely why I am looking at tools that integrate into, and enhance the existing setup.
Flag days make everyone unhappy.
From looking at the existing documentation and "pitch pages" for phabricator, I've had the constant feeling of getting overwhelmed. From the list of features you mentioned, I for one don't really see much of a difference between distinct Owners and Herald modules. If code review system has the concept of marking code ownership, then of course automatic notifications for incoming changes in that code should be standard practice.
But the moment CI gets implemented as core module of a code review system, I think it veers off into murky waters. You can't easily smoketest different CI systems. They are tightly coupled with release and deployment tooling, and carry the need for their own notifications. CI is a damn complex beast. I would rather have a very good CI with plugins for the extra functionality than a code repository / review system with a CI slapped on top. If you're going to integrate a CI into code repo (gitlab, I'm looking at you), then you have to aim really high. As in - if you find a regression, and commit a test for it, I would expect the integrated CI to be able to spin off an autonomous bisect run to detect exactly when the regression was introduced.
But let's get back to the subject at hand. The few things I feel are core to a really good code review system:
- Allows to automatically review feature branches, and keeps track of all the changes and comments locally; once the review is completed, the branch can be nuked, but the review remains as a historical item and contains full history
- Default reviewer groups. If a change set touches multiple parts of the code, the relevant people across teams ("owners", in phabricator I think) should all be marked as automatic reviewers.
- Related to the point above, the ability to automatically mark a changeset with "I'm looking at it" if you're doing the review in the web-UI. That would provide visibility for others in the reviewer group.
- Mail in support. Most people get code review comments delivered via email. It should be possible to reply to such a mail, and have the comments from the mail integrate back into the review thread visible in the web UI. If I could use my preferred mail client (mutt, in my case) to respond directly to code review threads, I would be quite a bit happier.
- Some kind of alerting mechanism for "problematic" reviews. If a change set takes ages to review, it usually means that either the change is too complex or too large - or possibly that the reviewer is overwhelmed.
- Again, only slightly related to the previous step: if you are going to integrate CI into a review system, it would be beneficial to see in the under-review section when a change has landed in master that would cause a merge conflict. The probability increases with code age, after all.
And that's just off the top of my head. I mentioned in the sibling comment that I realised RB is built as a product. That comes with its own baggage. In case of Phabricator, I see a an even more complex product. It appears to have so many tentacles that even Lovecraft could have been envious.
You're looking at GitLab and we're listening to you. We're aiming high with GitLab CI, the CI system on the planet or bust. Your suggestion for automatic bisecting is really interesting and we discussed it. The problem is that we can't see a way to do this without making it language dependent, this is not annotation for us.
Hey, thanks for the detailed feedback! It's always great to hear about use cases and what people need. Helps us shape things.
Something that's been core to our product's philosophy for a long time is to be extensible and to fit in with other tools. We're doing more to help there in 3.0, but we already have a very fleshed-out API and extension framework used to add on new features and capabilities, API, etc. The lack of an API for user account creation is clearly a pain point for you. I know I said we'd get that into 3.0, but I think I'm going to have to take that back.
I'm going to try to get it into the next 2.5.x instead.
> - Reviews need to be generated automatically when suitable trigger conditions are met. This means that there has to be a dedicated user account in RB that can create reviews on behalf of other users. Reviews also need to be automatically updated when new changes land in the branch under review.
You probably know this, but in case, or for anyone who doesn't, there's a permission that allows users to post or edit review requests on behalf of another user. I'll make sure that can be set in the API.
> Furthermore, I realise your permission model is tightly coupled with the underlying architecture, which itself builds on top of Django. Exposing the kind of functionality I ask for requires to expose and nest quite a few internal API layers. I do genuinely see why that has not been roadmap item so far. Codewise that is a big undertaking
Fortunately, large parts of this are already written for RBCommons.com (our Review Board SaaS) for our own use, and we should be able to port much of it over.
> Once you have native DVCS support and push-to-commit for fluid integration between other tools, RB has the ability become "just another tool". I see that as a good thing :)
It should be very cool :) Still a lot of work to get where we want to go, but it'll happen. Much of it already has, just not in a stable form yet.
> I haven't yet tried phabricator (evaluating it is on our infrastructure team's roadmap), but so far it looks like it tries to do too much.
I can't speak to their product. I believe we have different philosophies. Many tools choose to be like GitHub and do a little bit of everything under one product (which can be good if you aren't using other tools, or want to consolidate them all). We opt to focus solely on code review capabilities, trying to innovate in that area and plugging into other services/tools already being used. Pros and cons to both, but I certainly have my preference :)
Thanks again for the feedback! If you ever have more, or want to talk more about this outside Hacker News, definitely reach out! We have a dev-related list at https://groups.google.com/group/reviewboard-dev/, which I pay a lot of attention to, and others might have some good thoughts as well.
1) This has come up before. We'll probably add it. Most places that ask for it really are wanting something like an auth backend that ties into another service, like LDAP/ActiveDirectory (which are supported natively), a Single Sign-On service (which we'll be adding support for in Power Pack, our add-on product), or something more custom. For those who do want to add via the API, it's something I imagine we'll do in the next major release or two. I'll make sure it gets on the roadmap.
2) Review Board 4.0 (which is in development, alongside the upcoming 3.0) features DVCS support that will make it a lot easier to update patchsets and to keep them together as a whole without having to squash commits or create a review request per commit or anything else like that. Reviewers can view each commit individually, or the whole thing squashed, or a range squashed, and we'll help visualize how the history of those commits evolve with each update (such as adding new commits to HEAD, squashing commits together, etc.). It'll be really nice when it's done.
We're also looking at adding push-to-commit through our in-development Git (and eventually Mercurial) wrapper service, rb-gateway, so that you can push a branch there and have it appear on Review Board, and keep updating that branch to update the review request.
And, we're prototyping support for pull requests (first for GitHub, and then for others later), so that one can submit a pull request and have it basically appear in Review Board. This is still in the very early stages, but it's a priority. May not be relevant to your workflow, but thought I'd mention it, because that's asked a lot.
Again, Review Board 4.0 is when a lot of this will make an appearance, but I'll try to get account creation via the API to happen in 3.0. (Just added it to our roadmap.)