review.rst 17 KB

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899100101102103104105106107108109110111112113114115116117118119120121122123124125126127128129130131132133134135136137138139140141142143144145146147148149150151152153154155156157158159160161162163164165166167168169170171172173174175176177178179180181182183184185186187188189190191192193194195196197198199200201202203204205206207208209210211212213214215216217218219220221222223224225226227228229230231232233234235236237238239240241242243244245246247248249250251252253254255256257258259260261262263264265266267268269270271272273274275276277278279280281282283284285286287288289290291292293294295296297298299300301302303304305306307308309310311312313314315316317318319320321322323324325326327328329330331332333334335336337338339340341342343344345346347348349350351352353354355356357358359360361362363364365366367368369370371372373374375376377378379380381382383384385386387388389390391392393394395396397398399400401402403404405406407408409410411412413414415416417418419420421422423424425426427
  1. CMake Review Process
  2. ********************
  3. The following documents the process for reviewing and integrating changes.
  4. See `CONTRIBUTING.rst`_ for instructions to contribute changes.
  5. See documentation on `CMake Development`_ for more information.
  6. .. _`CONTRIBUTING.rst`: ../../CONTRIBUTING.rst
  7. .. _`CMake Development`: README.rst
  8. .. contents:: The review process consists of the following steps:
  9. Merge Request
  10. =============
  11. A user initiates the review process for a change by pushing a *topic
  12. branch* to his or her own fork of the `CMake Repository`_ on GitLab and
  13. creating a *merge request* ("MR"). The new MR will appear on the
  14. `CMake Merge Requests Page`_. The rest of the review and integration
  15. process is managed by the merge request page for the change.
  16. During the review process, the MR submitter should address review comments
  17. or test failures by updating the MR with a (force-)push of the topic
  18. branch. The update initiates a new round of review.
  19. We recommend that users enable the "Remove source branch when merge
  20. request is accepted" option when creating the MR or by editing it.
  21. This will cause the MR topic branch to be automatically removed from
  22. the user's fork during the `Merge`_ step.
  23. .. _`CMake Merge Requests Page`: https://gitlab.kitware.com/cmake/cmake/merge_requests
  24. .. _`CMake Repository`: https://gitlab.kitware.com/cmake/cmake
  25. Workflow Status
  26. ---------------
  27. `CMake GitLab Project Developers`_ may set one of the following labels
  28. in GitLab to track the state of a MR:
  29. * ``workflow:wip`` indicates that the MR needs additional updates from
  30. the MR submitter before further review. Use this label after making
  31. comments that require such updates.
  32. * ``workflow:in-review`` indicates that the MR awaits feedback from a
  33. human reviewer or from `Topic Testing`_. Use this label after making
  34. comments requesting such feedback.
  35. * ``workflow:nightly-testing`` indicates that the MR awaits results
  36. of `Integration Testing`_. Use this label after making comments
  37. requesting such staging.
  38. * ``workflow:expired`` indicates that the MR has been closed due
  39. to a period of inactivity. See the `Expire`_ step. Use this label
  40. after closing a MR for this reason.
  41. The workflow status labels are intended to be mutually exclusive,
  42. so please remove any existing workflow label when adding one.
  43. .. _`CMake GitLab Project Developers`: https://gitlab.kitware.com/cmake/cmake/settings/members
  44. Robot Review
  45. ============
  46. The "Kitware Robot" (``@kwrobot``) automatically performs basic checks on
  47. the commits proposed in a MR. If all is well the robot silently reports
  48. a successful "build" status to GitLab. Otherwise the robot posts a comment
  49. with its diagnostics. **A topic may not be merged until the automatic
  50. review succeeds.**
  51. Note that the MR submitter is expected to address the robot's comments by
  52. *rewriting* the commits named by the robot's diagnostics (e.g., via
  53. ``git rebase -i``). This is because the robot checks each commit individually,
  54. not the topic as a whole. This is done in order to ensure that commits in the
  55. middle of a topic do not, for example, add a giant file which is then later
  56. removed in the topic.
  57. Automatic Check
  58. ---------------
  59. The automatic check is repeated whenever the topic branch is updated.
  60. One may explicitly request a re-check by adding a comment with the
  61. following command among the `comment trailing lines`_::
  62. Do: check
  63. ``@kwrobot`` will add an award emoji to the comment to indicate that it
  64. was processed and also run its checks again.
  65. Automatic Format
  66. ----------------
  67. The automatic check will reject commits introducing source code not
  68. formatted according to ``clang-format``. One may ask the robot to
  69. automatically rewrite the MR topic branch with expected formatting
  70. by adding a comment with the following command among the
  71. `comment trailing lines`_::
  72. Do: reformat
  73. ``@kwrobot`` will add an award emoji to the comment to indicate that it
  74. was processed and also rewrite the MR topic branch and force-push an
  75. updated version with every commit formatted as expected by the check.
  76. Human Review
  77. ============
  78. Anyone is welcome to review merge requests and make comments!
  79. Please make comments directly on the MR page Discussion and Changes tabs
  80. and not on individual commits. Comments on a commit may disappear
  81. from the MR page if the commit is rewritten in response.
  82. Reviewers may add comments providing feedback or to acknowledge their
  83. approval. Lines of specific forms will be extracted during the `merge`_
  84. step and included as trailing lines of the generated merge commit message.
  85. Each review comment consists of up to two parts which must be specified
  86. in the following order: `comment body`_, then `comment trailing lines`_.
  87. Each part is optional, but they must be specified in this order.
  88. Comment Body
  89. ------------
  90. The body of a comment may be free-form `GitLab Flavored Markdown`_.
  91. See GitLab documentation on `Special GitLab References`_ to add links to
  92. things like issues, commits, or other merge requests (even across projects).
  93. Additionally, a line in the comment body may start with one of the
  94. following votes:
  95. * ``-1`` or ``:-1:`` indicates "the change is not ready for integration".
  96. * ``+1`` or ``:+1:`` indicates "I like the change".
  97. This adds an ``Acked-by:`` trailer to the `merge`_ commit message.
  98. * ``+2`` indicates "the change is ready for integration".
  99. This adds a ``Reviewed-by:`` trailer to the `merge`_ commit message.
  100. * ``+3`` indicates "I have tested the change and verified it works".
  101. This adds a ``Tested-by:`` trailer to the `merge`_ commit message.
  102. .. _`GitLab Flavored Markdown`: https://gitlab.kitware.com/help/user/markdown.md
  103. .. _`Special GitLab References`: https://gitlab.kitware.com/help/user/markdown.md#special-gitlab-references
  104. Comment Trailing Lines
  105. ----------------------
  106. Zero or more *trailing* lines in the last section of a comment may appear
  107. with the form ``Key: Value``. The first such line should be separated
  108. from a preceding `comment body`_ by a blank line. Any key-value pair(s)
  109. may be specified for human reference. A few specific keys have meaning to
  110. ``@kwrobot`` as follows.
  111. Comment Trailer Votes
  112. ^^^^^^^^^^^^^^^^^^^^^
  113. Among the `comment trailing lines`_ one may cast a vote using one of the
  114. following pairs followed by nothing but whitespace before the end of the line:
  115. * ``Rejected-by: me`` indicates "the change is not ready for integration".
  116. * ``Acked-by: me`` indicates "I like the change".
  117. This adds an ``Acked-by:`` trailer to the `merge`_ commit message.
  118. * ``Reviewed-by: me`` indicates "the change is ready for integration".
  119. This adds a ``Reviewed-by:`` trailer to the `merge`_ commit message.
  120. * ``Tested-by: me`` indicates "I have tested the change and verified it works".
  121. This adds a ``Tested-by:`` trailer to the `merge`_ commit message.
  122. Each ``me`` reference may instead be an ``@username`` reference or a full
  123. ``Real Name <user@domain>`` reference to credit someone else for performing
  124. the review. References to ``me`` and ``@username`` will automatically be
  125. transformed into a real name and email address according to the user's
  126. GitLab account profile.
  127. Comment Trailer Commands
  128. ^^^^^^^^^^^^^^^^^^^^^^^^
  129. Among the `comment trailing lines`_ authorized users may issue special
  130. commands to ``@kwrobot`` using the form ``Do: ...``:
  131. * ``Do: check`` explicitly re-runs the robot `Automatic Check`_.
  132. * ``Do: reformat`` rewrites the MR topic for `Automatic Format`_.
  133. * ``Do: test`` submits the MR for `Topic Testing`_.
  134. * ``Do: stage`` submits the MR for `Integration Testing`_.
  135. * ``Do: merge`` submits the MR for `Merge`_.
  136. See the corresponding sections for details on permissions and options
  137. for each command.
  138. Commit Messages
  139. ---------------
  140. Part of the human review is to check that each commit message is appropriate.
  141. The first line of the message should begin with one or two words indicating the
  142. area the commit applies to, followed by a colon and then a brief summary.
  143. Committers should aim to keep this first line short. Any subsequent lines
  144. should be separated from the first by a blank line and provide relevant, useful
  145. information.
  146. Area Prefix on Commit Messages
  147. ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  148. The appropriateness of the initial word describing the area the commit applies
  149. to is not something the automatic robot review can judge, so it is up to the
  150. human reviewer to confirm that the area is specified and that it is
  151. appropriate. Good area words include the module name the commit is primarily
  152. fixing, the main C++ source file being edited, ``Help`` for generic
  153. documentation changes or a feature or functionality theme the changes apply to
  154. (e.g. ``server`` or ``Autogen``). Examples of suitable first lines of a commit
  155. message include:
  156. * ``Help: Fix example in cmake-buildsystem(7) manual``
  157. * ``FindBoost: Add support for 1.64``
  158. * ``Autogen: Extended mocInclude tests``
  159. * ``cmLocalGenerator: Explain standard flag selection logic in comments``
  160. Referencing Issues in Commit Messages
  161. ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  162. If the commit fixes a particular reported issue, this information should
  163. ideally also be part of the commit message. The recommended way to do this is
  164. to place a line at the end of the message in the form ``Fixes: #xxxxx`` where
  165. ``xxxxx`` is the GitLab issue number and to separate it from the rest of the
  166. text by a blank line. For example::
  167. Help: Fix FooBar example robustness issue
  168. FooBar supports option X, but the example provided
  169. would not work if Y was also specified.
  170. Fixes: #12345
  171. GitLab will automatically create relevant links to the merge request and will
  172. close the issue when the commit is merged into master. GitLab understands a few
  173. other synonyms for ``Fixes`` and allows much more flexible forms than the
  174. above, but committers should aim for this format for consistency. Note that
  175. such details can alternatively be specified in the merge request description.
  176. Referencing Commits in Commit Messages
  177. ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  178. The preferred form for references to other commits is
  179. ``commit <commit> (<subject>, <date>)``, where:
  180. * ``<commit>``:
  181. If available, a tag-relative name of the commit produced by
  182. ``git describe --contains <commit-ish>``. Otherwise, the first
  183. 8-10 characters of the commit ``<hash>``.
  184. * ``<subject>``:
  185. The first line of the commit message.
  186. * ``<date>``:
  187. The author date of the commit, in its original time zone, formatted as
  188. ``CCYY-MM-DD``. ``git-log(1)`` shows the original time zone by default.
  189. Alternatively, the full commit ``<hash>`` may be used.
  190. Revising Commit Messages
  191. ^^^^^^^^^^^^^^^^^^^^^^^^
  192. Reviewers are encouraged to ask the committer to amend commit messages to
  193. follow these guidelines, but prefer to focus on the changes themselves as a
  194. first priority. Maintainers will also make a check of commit messages before
  195. merging.
  196. Topic Testing
  197. =============
  198. CMake has a `buildbot`_ instance watching for merge requests to test.
  199. `CMake GitLab Project Developers`_ may activate buildbot on a MR by
  200. adding a comment with a command among the `comment trailing lines`_::
  201. Do: test
  202. ``@kwrobot`` will add an award emoji to the comment to indicate that it
  203. was processed and also inform buildbot about the request. The buildbot
  204. user (``@buildbot``) will schedule builds and respond with a comment
  205. linking to the `CMake CDash Page`_ with a filter for results associated
  206. with the topic test request. If the MR topic branch is updated by a
  207. push a new ``Do: test`` command is needed to activate testing again.
  208. The ``Do: test`` command accepts the following arguments:
  209. * ``--stop``: clear the list of commands for the merge request
  210. * ``--clear``: clear previous commands before adding this command
  211. * ``--regex-include <arg>`` or ``-i <arg>``: only build on builders
  212. matching ``<arg>`` (a Python regular expression)
  213. * ``--regex-exclude <arg>`` or ``-e <arg>``: exclude builds on builders
  214. matching ``<arg>`` (a Python regular expression)
  215. Builder names follow the pattern ``project-host-os-buildtype-generator``:
  216. * ``project``: always ``cmake`` for CMake builds
  217. * ``host``: the buildbot host
  218. * ``os``: one of ``windows``, ``osx``, or ``linux``
  219. * ``buildtype``: ``release`` or ``debug``
  220. * ``generator``: ``ninja``, ``makefiles``, ``vs<year>``,
  221. or ``lint-iwyu-tidy``
  222. The special ``lint-<tools>`` generator name is a builder that builds
  223. CMake using lint tools but does not run the test suite (so the actual
  224. generator does not matter).
  225. .. _`buildbot`: http://buildbot.net
  226. .. _`CMake CDash Page`: https://open.cdash.org/index.php?project=CMake
  227. Integration Testing
  228. ===================
  229. The above `topic testing`_ tests the MR topic independent of other
  230. merge requests and on only a few key platforms and configurations.
  231. The `CMake Testing Process`_ also has a large number of machines
  232. provided by Kitware and generous volunteers that cover nearly all
  233. supported platforms, generators, and configurations. In order to
  234. avoid overwhelming these resources, they do not test every MR
  235. individually. Instead, these machines follow an *integration branch*,
  236. run tests on a nightly basis (or continuously during the day), and
  237. post to the `CMake CDash Page`_. Some follow ``master``. Most follow
  238. a special integration branch, the *topic stage*.
  239. The topic stage is a special branch maintained by the "Kitware Robot"
  240. (``@kwrobot``). It consists of the head of the MR target integration
  241. branch (e.g. ``master``) branch followed by a sequence of merges each
  242. integrating changes from an open MR that has been staged for integration
  243. testing. Each time the target integration branch is updated the stage
  244. is rebuilt automatically by merging the staged MR topics again.
  245. `CMake GitLab Project Developers`_ may stage a MR for integration testing
  246. by adding a comment with a command among the `comment trailing lines`_::
  247. Do: stage
  248. ``@kwrobot`` will add an award emoji to the comment to indicate that it
  249. was processed and also attempt to add the MR topic branch to the topic
  250. stage. If the MR cannot be added (e.g. due to conflicts) the robot will
  251. post a comment explaining what went wrong.
  252. Once a MR has been added to the topic stage it will remain on the stage
  253. until one of the following occurs:
  254. * The MR topic branch is updated by a push.
  255. * The MR target integration branch (e.g. ``master``) branch is updated
  256. and the MR cannot be merged into the topic stage again due to conflicts.
  257. * A developer or the submitter posts an explicit ``Do: unstage`` command.
  258. This is useful to remove a MR from the topic stage when one is not ready
  259. to push an update to the MR topic branch. It is unnecessary to explicitly
  260. unstage just before or after pushing an update because the push will cause
  261. the MR to be unstaged automatically.
  262. * The MR is closed.
  263. * The MR is merged.
  264. Once a MR has been removed from the topic stage a new ``Do: stage``
  265. command is needed to stage it again.
  266. .. _`CMake Testing Process`: testing.rst
  267. Resolve
  268. =======
  269. A MR may be resolved in one of the following ways.
  270. Merge
  271. -----
  272. Once review has concluded that the MR topic is ready for integration,
  273. `CMake GitLab Project Masters`_ may merge the topic by adding a comment
  274. with a command among the `comment trailing lines`_::
  275. Do: merge
  276. ``@kwrobot`` will add an award emoji to the comment to indicate that it
  277. was processed and also attempt to merge the MR topic branch to the MR
  278. target integration branch (e.g. ``master``). If the MR cannot be merged
  279. (e.g. due to conflicts) the robot will post a comment explaining what
  280. went wrong. If the MR is merged the robot will also remove the source
  281. branch from the user's fork if the corresponding MR option was checked.
  282. The robot automatically constructs a merge commit message of the following
  283. form::
  284. Merge topic 'mr-topic-branch-name'
  285. 00000000 commit message subject line (one line per commit)
  286. Acked-by: Kitware Robot <kwrobot@kitware.com>
  287. Merge-request: !0000
  288. Mention of the commit short sha1s and MR number helps GitLab link the
  289. commits back to the merge request and indicates when they were merged.
  290. The ``Acked-by:`` trailer shown indicates that `Robot Review`_ passed.
  291. Additional ``Acked-by:``, ``Reviewed-by:``, and similar trailers may be
  292. collected from `Human Review`_ comments that have been made since the
  293. last time the MR topic branch was updated with a push.
  294. The ``Do: merge`` command accepts the following arguments:
  295. * ``-t <topic>``: substitute ``<topic>`` for the name of the MR topic
  296. branch in the constructed merge commit message.
  297. Additionally, ``Do: merge`` extracts configuration from trailing lines
  298. in the MR description:
  299. * ``Topic-rename: <topic>``: substitute ``<topic>`` for the name of
  300. the MR topic branch in the constructed merge commit message.
  301. The ``-t`` option overrides this.
  302. .. _`CMake GitLab Project Masters`: https://gitlab.kitware.com/cmake/cmake/settings/members
  303. Close
  304. -----
  305. If review has concluded that the MR should not be integrated then it
  306. may be closed through GitLab.
  307. Expire
  308. ------
  309. If progress on a MR has stalled for a while, it may be closed with a
  310. ``workflow:expired`` label and a comment indicating that the MR has
  311. been closed due to inactivity.
  312. Contributors are welcome to re-open an expired MR when they are ready
  313. to continue work. Please re-open *before* pushing an update to the
  314. MR topic branch to ensure GitLab will still act on the association.