GitLab Code Reviews with Gemini Pro 1.5

Contents:

• Introduction
• Prerequisites
• GCP Setup
• GitLab Configuration
• The Code
• Prompt Engineering
• Testing and Results

Introducing My AI Coding Companion: Google Gemini Pro

Have you ever wished you had a personal AI assistant to review your code? As a student developer, I often felt like I was missing out on the benefits of collaborative code reviews. While experimenting with Google Gemini Pro, I was blown away by its ability to interpret, summarize, and provide feedback on complex information. That’s when I had an idea: why not use Gemini’s powerful capabilities to revolutionize my coding experience?

To bridge the gap between education and professional workflows, I decided to experiment with an innovative solution: leveraging Google Gemini Pro to automate code reviews and documentation updates within my GitLab workflow. By connecting GitLab webhooks to a Google Cloud Function that calls the Vertex AI API, I envisioned a system where Gemini would analyze my code changes, generate insightful feedback, and even automatically update my project’s wiki. The results were remarkable… but I’ll get to those later.

Prerequisites: Tools and Accounts You’ll Need

To follow along with this tutorial, you’ll need:

• A GitLab Account: If you don’t have one, you can sign up for a free account on the GitLab website.
• A Google Cloud Platform (GCP) Project: This will be used to deploy your Cloud Function and access the Vertex AI API. If you’re new to GCP, they offer a free trial with credits to get you started.
• Basic Git Knowledge: You’ll need a basic understanding of how to use Git to manage your code repository, commit changes, and create merge requests.
• Python Programming Experience: The Cloud Function we’ll be building will be written in Python, so some familiarity with the language is helpful.

GCP Setup: Bringing Gemini to the Cloud

To deploy our Gemini-powered code reviewer, we’ll leverage Google Cloud Functions (GCF), a serverless platform that lets you run code without managing servers. This will make our integration easily scalable and cost-effective.

Enable APIs:

• Head to the Google Cloud Console.
• Ensure the following APIs are enabled for your project
  – Cloud Functions API
  – Secret Manager API
  – Vertex AI API

Create a Cloud Function:

• Name your function (e.g., “gitlab-gemini-code-review”).
• Choose a region (e.g., us-central1).
• Select “2nd gen” environment.
• Set the trigger type to “HTTP.”
• Increase the allocated memory to 512 MiB in the Runtime settings.

Create the Secret to authorize GitLab communication:

• Go to the “Secret Manager” section.
• Click on “Create Secret”.
• Name your secret (e.g., GITLAB_WEBHOOK_SECRET_ID).
• Provide a strong, random string of characters as the secret value.
• Click “Create Secret.”

We’ll now set up a user to act as a bot, project access token, and webhook in GitLab to allow it to communicate with our Cloud Function.

GitLab Configuration:

To enable Gemini to interact with your GitLab project, follow these steps:

Create a New GitLab User:

Note: I advise turning off all email notifications for this bot.

Add the Bot User to Your Project:

Why the “Developer” Role?
This role provides Gemini with the necessary permissions for code 
review (posting comments on merge requests) and wiki updates 
(editing pages) while still adhering to the principle of least 
privilege. Feel free to explore all the roles available to get a 
better understanding of them.

Creating a Personal Access Token:

Note: If you leave the expiration date blank, GitLab will automatically
set the PAT to expire in one year.

Creating a Project Webhook:

• Go to your GitLab project’s settings and click on “Webhooks.”
• Click “Add webhook.”
• In the “URL” field, paste the URL of your deployed Cloud Function. This is where GitLab will send the notifications.
• In the “Secret Token” field, enter the secret token you stored in Google Cloud Secret Manager (remember: GITLAB_WEBHOOK_SECRET_ID). This ensures that only authorized requests reach your function.
• Important: Select the “Merge request events” trigger. This will notify your Cloud Function whenever a merge request is created, updated, or merged.
• Click “Add webhook.”

Great! Now that we have our environment set up, let’s dive into the code that makes this integration possible.

The Code: The Nitty Gritty

The heart of this project lies in the Python code that powers the Google Cloud Function. This code orchestrates the entire integration process, from handling GitLab webhooks to interacting with Gemini and updating your GitLab project.

Repository:

ingenuityAi-robotics / gitlab-gemini-code-review · GitLab

GitLab.com

GitLab API, Webbhook, and Python Package Documentation:

• https://docs.gitlab.com/ee/api/merge_requests.html
• https://docs.gitlab.com/ee/user/project/integrations/webhook_events.html#merge-request-events
• https://python-gitlab.readthedocs.io/en/stable/api/gitlab.v4.html#gitlab.v4.objects.Project.wikis

Core Functionalities:

Configuring the GitLab Webhook: Picking Your Triggers

The GitLab webhook is the bridge that allows your project to communicate with the Gemini-powered Cloud Function. It’s essential to configure it correctly to trigger actions only when necessary.

Choosing the Right Events:

GitLab webhooks offer a wide range of events to trigger actions. For this project, we’ll focus on Merge Request Events. These are triggered whenever a merge request is opened, updated, or merged. We’ll use this to initiate both code reviews and wiki updates.

Handling Merge Request Actions:

Even within “Merge Request Events,” GitLab provides more granular information through the object_attributes.action field in the webhook payload. This field tells us the specific action that triggered the webhook. We’ll handle the following actions:

Note: the oldrev attribute will only be present with a merge request update
if new code has been submitted. 

Understanding GitLab Diffs: A Closer Look

In GitLab, a “diff” represents the changes between two versions of a file or project. It’s essentially a way to visualize what lines have been added, modified, or removed. While diffs are fundamental to code review and version control, their structure in GitLab’s API can be a bit tricky to grasp.

GitLab’s Diff Structure:

• Merge Request Diffs (mr.diffs.list()): When you call this method on a merge request object in the python-gitlab library, you get a list of preview objects that may not contain all the attributes for the Diff. We iterate over the ProjectMergeRequestDiff objects and use their diff attributes to get the list of raw diff string hunks for each file.
• Single Commit Diffs (diff_obj.diffs): This is a property that contains the raw text of the diff for the changed file. This is essentially a big string of text that can be parsed to extract the changes on a line-by-line basis. This will be a JSON list of all the diff hunks for that particular commit.
• Line-Level Detail: The raw diff text includes line numbers and indicators (+ for added lines, - for removed lines) to show the specific changes within the file.

Example Diff Chunk:

@@ -1,3 +1,4 @@
-This is the old line 1.
+This is the new line 1.
 This line is unchanged.
-This is the old line 3.
+This is the modified line 3.
+This is a new line 4.

Prompt Engineering: Guiding Gemini’s Analysis

When I first started coding this integration, I anticipated a complex process of breaking down each merge request into individual diffs (chunks of changes within files) and feeding them to Gemini one by one. I envisioned having to carefully parse and analyze Gemini’s responses to extract feedback for each line of code. However, I soon discovered that Gemini was far more capable than I had initially imagined!

Gemini’s Diff-Parsing Prowess:

After some experimentation, I realized that I could simply give Gemini the entire list of diffs as a JSON array and instruct it on the structure of this data. This meant I could bypass the intricate diff-parsing logic I had initially planned. Gemini, with its advanced natural language understanding, was able to interpret the diff formats and identify the relevant parts of the code for review.

System Instructions (Priming) vs. Prompt: Guiding the AI’s Behavior

When working with language models like Gemini, you essentially have two ways to communicate your intentions and steer the AI’s response:

System Instructions:

• What they are: High-level directives that set the overall behavior, tone, and style of the AI.
• Purpose: These instructions provide the AI with a framework for understanding its role and how it should approach the task.
• Where they’re used: Typically set once when you initialize the GenerativeModel object in your code.
• Persistence: The system instructions remain in effect for the entire session with the model, influencing all subsequent responses.

Prompt:

• What it is: The specific input you provide to the AI for each request.
• Purpose: It presents the task or question you want the AI to address in that particular instance.
• How it’s used: Sent with each call to the model.generate_content() method.
• Contextual: The prompt provides the immediate context for the AI’s response.

I experimented with both having minimal system instructions and a verbose prompt, as well as, having verbose system instructions and a minimal prompt. My initial results were much better with the system instructions being very verbose and exact.

The Instructions:

"""You are a helpful code reviewer. Your mission is to review the code changes for a ROS2 package and provide feedback based on the changes.
            
    The Changess will be provided as a JSON Array of Changes.
    
    Code Change Structure:
    * **`diff`**: The diff content showing the changes made to the file (see Diff Header Format.)
    * **`new_path`**: The path to the modified or new file.
    * **`old_path`**: The original path to the file (if it was renamed or moved).
    * **`a_mode`**: The mode of the file before the change.
    * **`b_mode`**: The mode of the file after the change.
    * **`new_file`**: Whether the file is a new file or not.
    * **`renamed_file`**: Whether the file was renamed or moved.
    * **`deleted_file`**: Whether the file was deleted or not.
    * **'generated_file`**: Whether the file was generated by AI or not.
    
    **Diff Format**: 
    
    The diff header will be in the following format:
    ```Diff Header Format
    @@ -[start line],[number of lines] +[start line],[number of lines] @@
    ```
    * **`@@`**: This is the opening encapsulation to identify the diff header.
    * **`-`**: This sign indicates the next set of lines are lines from the original file.
    * **`+`**: This sign indicates the next set of lines are lines from the modified file. 
    * **`[start line],[number of lines]`**: This indicates the starting line of the the change and the number of lines affected.
    * **`@@`**: This is the closing encapsulation to identify the diff header.
    
    ```Diff Header Example
    @@ -15,8 +15,10 @@
    ```
    This means:
    * 8 lines starting from line 15 (one-indexed) in the original file.
    * 10 lines starting from line 15 (one-indexed) in the modified file.
    
    The diff body will be in the following format:
    * **`-`**: If the line starts with the `-` sign, it represents a line removed from the original file.
    * **`+`**: If the line starts with the `+` sign, it represents a line added in the modified file.
    * ** no (`-` or `+`)**: A line that does not start with a `-` or a `+` sign belongs to both files and did not change. 

    When reviewing the code, please focus on the following aspects:

    * **Correctness:** Are there any potential errors, bugs, or logic flaws in the code?
    * **Typos and Formatting:** Are there any typos, grammatical errors, or formatting inconsistencies that should be corrected?
    * **Maintainability:** Is the code easy to understand, well-structured, and documented?  Are there opportunities to simplify or refactor the code?
    * **Performance:** Are there any potential performance bottlenecks or areas where efficiency could be improved?
    * **Security:** Are there any security vulnerabilities or potential risks (e.g., input validation, error handling)?
    * **ROS2 Best Practices:** Does the code follow ROS2 conventions and best practices (e.g., node naming, parameter usage, message types)?

    You **MUST** label comments according to importance:

    * **minor**: For minor issues like typos or formatting inconsistencies.
    * **moderate**: For issues that affect code quality but aren't critical.
    * **major**: For significant issues that could lead to errors or problems.
    * **critical**: For critical errors or vulnerabilities that require immediate attention.

    Provide a detailed code review in JSON format with the following headers `for each relevant line` and file:

    * **`new_line`**: the line in the new file that was added (started with a `+` sign) that the comment applies to incremented by one. 
    * **`old_line`**: the line in the old file that was modified (started with a `-` sign) that the comment applies to incremented by one.
    * **`new_file_path`**: The path to the file where the change occurred (e.g., "README.md").
    * **`old_file_path`**: The original path to the file (if it was renamed or moved).
    * **`comment`**: A concise description of the issue, potential improvement, or observation regarding ROS2 best practices in Markdown format.
    * **`severity`**: The severity of the issue ('minor', 'moderate', 'major', 'critical').
    
    **Special Instructions:**

    * To comment on a line that was added, the `new_line` attribute should be filled, and the `old_line` attribute should be `null`.
    * To comment on a line that was removed, the `new_line` attribute should be `null`, and the `old_line` attribute should be filled.
    * To comment on a line that was **not changed**, both the `new_line` and `old_line` attributes should be filled, and they **must have the same line number value**.
    * Only include rows for lines that require feedback, but be sure to review all diffs and files. 

    **Additional Emphasis**

    * Please ensure strict adherence to these special instructions regarding how `new_line` and `old_line` are populated based on whether a line was added, removed, or unchanged."""

Why Markdown?

Markdown is a lightweight markup language that allows you to add formatting elements to plain text. It’s easy to read and write, making it a great choice for communicating with language models like Gemini. Here’s why we should use it in our prompts and instructions:

• Structure: Markdown provides a clear structure for the prompt, making it easier for both humans and AI to understand the different sections and expectations.
• Emphasis: We can use Markdown’s formatting options (e.g., headings, bold, italics) to highlight important aspects of the task and guide Gemini’s focus.
• Code Blocks: Markdown’s code block syntax (```) allows us to delineate code snippets within the prompt, making it easier for Gemini to recognize and analyze them.

In other words, just like humans, Gemini perceives emphasis and structure to understand context just as much as it does the words given.

Parts and Pieces:

Diff Structure Explanation: It explains to Gemini that the input is a JSON array of diffs, each containing keys like diff (the raw diff text), new_path, and old_path.

Desired Output Format: It instructs Gemini to provide the code review feedback in a specific JSON format, including fields like line_number_start, line_number_end, file_path, and comment. This structured output makes it easy for the Cloud Function to process the feedback later.

Code Review Focus Areas: It outlines the specific aspects of the code that Gemini should analyze, such as correctness, maintainability, performance, security, and adherence to ROS2 best practices.

Severity Labels: It defines a set of severity labels (minor, moderate, major, critical) and instructs Gemini to use them to categorize its feedback. This helps prioritize the most important issues and makes the review more actionable.

Additional Instructions: It provides additional guidance to Gemini, such as requesting concrete suggestions for improvement, examples, and code snippets where relevant.

By crafting comprehensive instructions, I was able to leverage Gemini’s capabilities to generate detailed, structured, and actionable code reviews without having to write extensive parsing logic myself. This saved me valuable development time and made the integration much more efficient.

Testing and Results

The moment of truth arrived: testing the integration. I created merge requests with errors, inconsistencies, and areas for optimization.

Code Reviews: Impressive Results

Gemini’s code reviews were impressive. It consistently found errors, suggested improvements, and flagged potential performance issues. The feedback was clear, concise, and actionable, helping me improve my code.

• Severity Ranking and JSON Consistency: The severity ranking system was valuable, helping prioritize issues. Gemini’s consistent JSON output enabled seamless integration with my Cloud Function.
• Final Review Summary: The final part of the code review, is for Gemini to post a summary of its findings. This was always consistent and did as asked.

GitLab Integration: A Minor Challenge

I encountered a minor issue with posting comments on unchanged lines. GitLab’s API seemed to throw an error, preventing Gemini’s feedback from being displayed. I will work on implementing multiline discussions and have Gemini generate the line numbers to use.

Conclusion: The Future of Code Reviews

This integration demonstrates AI’s potential in code reviews. Automating routine checks frees up developer time, allowing them to focus on creating software. While not a replacement for human reviewers, Gemini catches many issues, potentially saving senior developers significant time.

The future of code reviews lies in the collaboration between human expertise and AI capabilities. By embracing AI tools, we can create a more efficient and productive development experience.