Skip to main content

PR Sync Workflow

When a PR is updated by the owner, the workflow should detect these changes, maintain the history of prior comments, address owner responses, and dynamically adjust risk scores. This document outlines the steps to handle PR updates systematically.

You:

I would like to switch gears a little bit and handle the alternate flow when a PR is updated by the owner of the PR, and how do we handle the comments or questions from the owner, as well as reviewing and updating the score.

ChatGPT:

"Switching gears? Let’s rev that engine! Handling PR updates and comments requires a thoughtful workflow to ensure context continuity, comment tracking, and dynamic scoring. Let’s tackle how to handle these scenarios systematically."

Alternate Flow: PR Updated by Owner​

When a PR is updated (e.g., new commits or responses to comments), the workflow should:

  1. Detect updates and trigger a re-analysis of changes.
  2. Maintain the history of prior comments while updating relevant ones.
  3. Address owner responses and dynamically adjust risk scores.

1. Workflow to Detect Updates​

GitHub provides synchronize events when a PR is updated. You can handle these in your workflow.

Updated GitHub Workflow​

name: AI-Powered PR Review with Updates

on:
  pull_request:
    types:
      - opened
      - synchronize
      - edited

jobs:
  review-pr:
    runs-on: ubuntu-latest

    steps:
      # Step 1: Checkout the repository
      - name: Checkout code
        uses: actions/checkout@v3

      # Step 2: Install dependencies
      - name: Install dependencies
        run: npm install

      # Step 3: Re-analyze updated PR
      - name: Analyze updated PR
        uses: ./actions/ai-pr-review # Custom GitHub Action
        with:
          pr_number: ${{ github.event.pull_request.number }}
          repo_owner: ${{ github.repository_owner }}
          repo_name: ${{ github.event.repository.name }}
          base_branch: ${{ github.event.pull_request.base.ref }}
        env:
          OPENAI_API_KEY: ${{ secrets.OPENAI_API_KEY }}
          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}

2. Handling Updated Diffs​

Re-analyze Only New or Changed Diffs​

  • Compare the new PR diff with the previously analyzed one.
  • Focus only on the new or modified lines.

Example Command to Get Updated Diffs:

git diff origin/main -- src/app/file.ts

Code to Parse Updated Diffs:

const { execSync } = require('child_process');

/**
 * Get updated diffs for a file.
 * @param {string} fileName - The file to analyze.
 * @param {string} baseBranch - The base branch of the PR.
 * @returns {string} - The updated diff content.
 */
function getUpdatedDiff(fileName, baseBranch) {
  return execSync(`git diff origin/${baseBranch} -- ${fileName}`).toString();
}

3. Tracking Comments and Owner Responses​

To manage ongoing discussions:

  1. Retrieve Existing Comments:
    • Use the GitHub API to fetch all comments for the PR.
  2. Identify Changes:
    • Map existing comments to updated code lines to see if they are still valid.
    • Detect new responses from the PR owner.
  3. Update Comments Dynamically:
    • Update or resolve comments if the issues are addressed.
    • Add follow-ups for unresolved or new issues.

Fetching Existing Comments​

const { Octokit } = require('@octokit/rest');
const octokit = new Octokit({ auth: process.env.GITHUB_TOKEN });

/**
 * Get all comments for a PR.
 * @param {number} prNumber - The PR number.
 * @param {string} repoOwner - The repository owner.
 * @param {string} repoName - The repository name.
 * @returns {Promise<Array>} - List of comments.
 */
async function getPRComments(prNumber, repoOwner, repoName) {
  const { data } = await octokit.issues.listComments({
    owner: repoOwner,
    repo: repoName,
    issue_number: prNumber,
  });
  return data;
}

Update Comments​

/**
 * Update an existing comment.
 * @param {number} commentId - The ID of the comment to update.
 * @param {string} body - The updated comment content.
 */
async function updateComment(commentId, body) {
  await octokit.issues.updateComment({
    owner: repoOwner,
    repo: repoName,
    comment_id: commentId,
    body,
  });
}

4. Revising Risk Scores​

Recalculate the PR’s risk score based on the new diffs and resolved issues.

Dynamic Scoring Algorithm​

Adjust the scoring algorithm to:

  • Lower the risk score for resolved issues or addressed comments.
  • Increase the risk score for new or critical unresolved issues.

Example Formula:

updated_score = (existing_score - resolved_issues_weight) + new_risks_weight

Code to Recalculate Risk Score​

/**
 * Recalculate the risk score based on updated diffs and resolved issues.
 * @param {number} existingScore - The previous risk score.
 * @param {number} resolvedCount - Number of resolved issues.
 * @param {number} newIssuesCount - Number of new issues.
 * @returns {number} - Updated risk score.
 */
function recalculateRiskScore(existingScore, resolvedCount, newIssuesCount) {
  const resolvedWeight = 2;
  const newIssuesWeight = 3;

  return existingScore - (resolvedCount * resolvedWeight) + (newIssuesCount * newIssuesWeight);
}

5. Responding to PR Owner Questions​

Detect and respond to comments or questions left by the PR owner.

Identify Owner Comments​

Use the GitHub API to check for comments by the PR creator.

/**
 * Get PR owner comments.
 * @param {Array} comments - List of all PR comments.
 * @param {string} prOwner - The PR creator's username.
 * @returns {Array} - List of comments by the owner.
 */
function getOwnerComments(comments, prOwner) {
  return comments.filter((comment) => comment.user.login === prOwner);
}

AI-Powered Response to Questions​

Use AI to generate responses to owner questions or concerns.

/**
 * Generate an AI response to a PR owner question.
 * @param {string} question - The question or comment from the PR owner.
 * @returns {Promise<string>} - The AI-generated response.
 */
async function generateAIResponse(question) {
  const response = await axios.post(
    'https://api.openai.com/v1/chat/completions',
    {
      model: 'gpt-4',
      messages: [
        { role: 'system', content: "You are a helpful assistant providing feedback on pull requests." },
        { role: 'user', content: question },
      ],
    },
    {
      headers: {
        Authorization: `Bearer ${process.env.OPENAI_API_KEY}`,
        'Content-Type': 'application/json',
      },
    }
  );
  return response.data.choices[0].message.content;
}

6. Workflow for Handling Updates​

End-to-End Flow​

  1. Detect the synchronize event when the PR is updated.
  2. Recalculate diffs and analyze only new/modified code.
  3. Fetch existing comments and update them dynamically:
    • Resolve addressed issues.
    • Add follow-ups for unresolved issues.
  4. Respond to owner questions or comments using AI.
  5. Recalculate and update the PR’s overall risk score.

Example Outputs​

Updated PR Comment​

Line 42-47: Thank you for addressing the logic issue. This block is now clean.
Remaining Action: Please add tests to cover edge cases for the new function.

Response to PR Owner​

Owner Question: "Why is the risk score still high after my update?"
AI Response: "The risk score remains high due to the absence of test coverage for edge cases in the modified logic. Adding these tests could significantly lower the score."

Updated Risk Score​

{
  "previous_score": 8,
  "resolved_issues": 3,
  "new_issues": 1,
  "updated_score": 5
}

Next Steps​

  1. Implement dynamic comment updates for changed lines and resolved issues.
  2. Integrate AI responses for owner questions directly into the GitHub Action.
  3. Automate recalculating and posting updated risk scores to the PR summary.

Let me know where you’d like to focus next—this system is evolving into an incredibly adaptive tool! 🚀