Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add processSavePullRequest function and integrate it into the savePullRequestTask #869

Merged
merged 1 commit into from
Mar 13, 2025

Conversation

hoshinotsuyoshi
Copy link
Member

@hoshinotsuyoshi hoshinotsuyoshi commented Mar 12, 2025

Issue

  • resolve:

Why is this change needed?

What would you like reviewers to focus on?

Testing Verification

What was done

🤖 Generated by PR Agent at a9071cb

  • Introduced processSavePullRequest function for PR data handling.
  • Integrated savePullRequestTask with webhook and database operations.
  • Enhanced GitHub API file change handling with detailed statuses.
  • Improved error handling and logging in webhook processing.

Detailed Changes

Relevant files
Enhancement
6 files
route.ts
Refactor webhook handling and integrate PR save task         
+34/-12 
api.ts
Extend file change handling with detailed statuses             
+10/-1   
processSavePullRequest.ts
Add `processSavePullRequest` function for PR data processing
+83/-0   
jobs.ts
Integrate `processSavePullRequest` into `savePullRequestTask`
+30/-10 
index.ts
Update types to support optional projectId                             
+2/-2     
github.ts
Enhance `FileChange` type with detailed statuses and patch
+9/-1     
Documentation
2 files
pull-request.ts
Mark `handlePullRequest` as deprecated                                     
+1/-0     
flow.md
Document updated PR save and review workflow                         
+8/-1     

Additional Notes


Need help?
  • Type /help how to ... in the comments thread for any questions about Qodo Merge usage.
  • Check out the documentation for more information.
  • @hoshinotsuyoshi hoshinotsuyoshi self-assigned this Mar 12, 2025
    Copy link

    changeset-bot bot commented Mar 12, 2025

    ⚠️ No Changeset found

    Latest commit: a9071cb

    Merging this PR will not cause a version bump for any packages. If these changes should not result in a new version, you're good to go. If these changes should result in a version bump, you need to add a changeset.

    This PR includes no changesets

    When changesets are added to this PR, you'll see the packages that this PR includes changesets for and the associated semver types

    Click here to learn what changesets are, and how to add one.

    Click here if you're a maintainer who wants to add a changeset to this PR

    Copy link

    vercel bot commented Mar 12, 2025

    The latest updates on your projects. Learn more about Vercel for Git ↗︎

    Name Status Preview Comments Updated (UTC)
    liam-docs ✅ Ready (Inspect) Visit Preview 💬 Add feedback Mar 13, 2025 8:25am
    liam-erd-sample ✅ Ready (Inspect) Visit Preview 💬 Add feedback Mar 13, 2025 8:25am
    3 Skipped Deployments
    Name Status Preview Comments Updated (UTC)
    test-liam-docs ⬜️ Ignored (Inspect) Mar 13, 2025 8:25am
    test-liam-erd-sample ⬜️ Ignored (Inspect) Mar 13, 2025 8:25am
    test-liam-erd-web ⬜️ Ignored (Inspect) Mar 13, 2025 8:25am

    Copy link

    supabase bot commented Mar 12, 2025

    Updates to Preview Branch (save-function-1) ↗︎

    Deployments Status Updated
    Database Thu, 13 Mar 2025 08:15:33 UTC
    Services Thu, 13 Mar 2025 08:15:33 UTC
    APIs Thu, 13 Mar 2025 08:15:33 UTC

    Tasks are run on every commit but only new migration files are pushed.
    Close and reopen this PR if you want to apply changes from existing seed or migration files.

    Tasks Status Updated
    Configurations Thu, 13 Mar 2025 08:15:33 UTC
    Migrations Thu, 13 Mar 2025 08:15:34 UTC
    Seeding Thu, 13 Mar 2025 08:15:34 UTC
    Edge Functions Thu, 13 Mar 2025 08:15:34 UTC

    View logs for this Workflow Run ↗︎.
    Learn more about Supabase for Git ↗︎.

    Copy link
    Contributor

    qodo-merge-pro-for-open-source bot commented Mar 13, 2025

    CI Feedback 🧐

    (Feedback updated until commit 21fe4c6)

    A test triggered by this PR failed. Here is an AI-generated analysis of the failure:

    Action: Deploy (app, VERCEL_PROJECT_ID_ERD_WEB)

    Failed stage: Run prepare command [❌]

    Failure summary:

    The build action failed due to a type error in the TypeScript code. Specifically:

  • In file src/functions/processSavePullRequest.ts (line 47), there's a type mismatch where a bigint
    value (repository.installationId) is being passed to a function that expects a number parameter.
  • The error occurs when calling the getPullRequestFiles function with repository.installationId as the
    first argument.
  • Additionally, there was a warning about a Node.js module (async_hooks) being loaded which is not
    supported in the Edge Runtime.

  • Relevant error logs:
    1:  ##[group]Operating System
    2:  Ubuntu
    ...
    
    180:  Lockfile is up to date, resolution step is skipped
    181:  Progress: resolved 1, reused 0, downloaded 0, added 0
    182:  Packages: +1720
    183:  ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
    184:  Progress: resolved 1720, reused 1116, downloaded 0, added 0
    185:  Progress: resolved 1720, reused 1702, downloaded 0, added 459
    186:  Progress: resolved 1720, reused 1702, downloaded 0, added 1320
    187:  Progress: resolved 1720, reused 1702, downloaded 0, added 1720, done
    188:  WARN  Failed to create bin at /home/runner/work/liam/liam/frontend/apps/erd-sample/node_modules/.bin/liam. ENOENT: no such file or directory, open '/home/runner/work/liam/liam/frontend/packages/cli/dist-cli/bin/cli.js'
    ...
    
    194:  + syncpack 13.0.0
    195:  + turbo 2.1.2
    196:  + vercel 41.4.0
    197:  frontend/apps/docs postinstall$ fumadocs-mdx
    198:  frontend/apps/docs postinstall: [MDX] types generated
    199:  frontend/apps/docs postinstall: Done
    200:  frontend/apps/app postinstall$ cp ../../packages/db-structure/node_modules/@ruby/prism/src/prism.wasm prism.wasm
    201:  frontend/apps/app postinstall: Done
    202:  WARN  Failed to create bin at /home/runner/work/liam/liam/frontend/apps/erd-sample/node_modules/.bin/liam. ENOENT: no such file or directory, open '/home/runner/work/liam/liam/frontend/apps/erd-sample/node_modules/@liam-hq/cli/dist-cli/bin/cli.js'
    ...
    
    296:  Wrote /home/runner/work/liam/liam/frontend/packages/erd-core/src/styles/variables.css.d.ts
    297:  Wrote /home/runner/work/liam/liam/frontend/packages/erd-core/src/styles/globals.css.d.ts
    298:  Wrote /home/runner/work/liam/liam/frontend/packages/erd-core/src/features/erd/components/ERDRenderer/ERDRenderer.module.css.d.ts
    299:  Wrote /home/runner/work/liam/liam/frontend/packages/erd-core/src/features/erd/components/ERDContent/ERDContent.module.css.d.ts
    300:  Wrote /home/runner/work/liam/liam/frontend/packages/erd-core/src/features/erd/components/ERDRenderer/Toolbar/DesktopToolbar.module.css.d.ts
    301:  Wrote /home/runner/work/liam/liam/frontend/packages/erd-core/src/features/erd/components/ERDRenderer/TableDetailDrawer/TableDetailDrawer.module.css.d.ts
    302:  Wrote /home/runner/work/liam/liam/frontend/packages/erd-core/src/features/erd/components/ERDRenderer/RelationshipEdgeParticleMarker/RelationshipEdgeParticleMarker.module.css.d.ts
    303:  Wrote /home/runner/work/liam/liam/frontend/packages/erd-core/src/features/erd/components/ERDRenderer/LeftPane/LeftPane.module.css.d.ts
    304:  Wrote /home/runner/work/liam/liam/frontend/packages/erd-core/src/features/erd/components/ERDRenderer/ErrorDisplay/ParseErrorDisplay.module.css.d.ts
    305:  Wrote /home/runner/work/liam/liam/frontend/packages/erd-core/src/features/erd/components/ERDRenderer/ErrorDisplay/NetworkErrorDisplay.module.css.d.ts
    306:  Wrote /home/runner/work/liam/liam/frontend/packages/erd-core/src/features/erd/components/ERDRenderer/ErrorDisplay/ErrorDisplay.module.css.d.ts
    ...
    
    398:  . build:vite:  * [new branch]        avoid-semicolon-logic-test -> origin/avoid-semicolon-logic-test
    399:  . build:vite:  * [new branch]        before-promptfoo-experiment -> origin/before-promptfoo-experiment
    400:  . build:vite:  * [new branch]        changeset-invest1        -> origin/changeset-invest1
    401:  . build:vite:  * [new branch]        changeset-invest3        -> origin/changeset-invest3
    402:  . build:vite:  * [new branch]        changeset-release/changeset-invest1 -> origin/changeset-release/changeset-invest1
    403:  . build:vite:  * [new branch]        check-github-action      -> origin/check-github-action
    404:  . build:vite:  * [new branch]        cli-schema-json-for-local -> origin/cli-schema-json-for-local
    405:  . build:vite:  * [new branch]        comment-post-job         -> origin/comment-post-job
    406:  . build:vite:  * [new branch]        console-error-1-draft    -> origin/console-error-1-draft
    ...
    
    411:  . build:vite:  * [new branch]        devin/1738828298-add-multiline-comment-support -> origin/devin/1738828298-add-multiline-comment-support
    412:  . build:vite:  * [new branch]        devin/1739501885-improve-navigation-test-waits -> origin/devin/1739501885-improve-navigation-test-waits
    413:  . build:vite:  * [new branch]        devin/1740113652-update-changeset-docs -> origin/devin/1740113652-update-changeset-docs
    414:  . build:vite:  * [new branch]        devin/1741661851-optimize-github-config-validation -> origin/devin/1741661851-optimize-github-config-validation
    415:  . build:vite:  * [new branch]        devin/1741663537-github-env-validation-at-boot -> origin/devin/1741663537-github-env-validation-at-boot
    416:  . build:vite:  * [new branch]        devin/1741825670-postgresql-parser-test -> origin/devin/1741825670-postgresql-parser-test
    417:  . build:vite:  * [new branch]        devin/1741852989-update-auth-redirect -> origin/devin/1741852989-update-auth-redirect
    418:  . build:vite:  * [new branch]        enhance-the-highlighting-of-related-table-nodes-when-hovering-usememo-1 -> origin/enhance-the-highlighting-of-related-table-nodes-when-hovering-usememo-1
    419:  . build:vite:  * [new branch]        error-message-erd-network-error -> origin/error-message-erd-network-error
    ...
    
    454:  . build:vite:  * [new branch]        primary-key-icon-2       -> origin/primary-key-icon-2
    455:  . build:vite:  * [new branch]        promptfoo-go-1           -> origin/promptfoo-go-1
    456:  . build:vite:  * [new branch]        reduce-set-node-calling-chance-3 -> origin/reduce-set-node-calling-chance-3
    457:  . build:vite:  * [new branch]        refactor-active-highlight -> origin/refactor-active-highlight
    458:  . build:vite:  * [new branch]        refactor-to-testcases    -> origin/refactor-to-testcases
    459:  . build:vite:  * [new branch]        refactor_edge_cardinality_by_using_custom_mark -> origin/refactor_edge_cardinality_by_using_custom_mark
    460:  . build:vite:  * [new branch]        refactor_hidden_node_handle -> origin/refactor_hidden_node_handle
    461:  . build:vite:  * [new branch]        release-debug-branch     -> origin/release-debug-branch
    462:  . build:vite:  * [new branch]        revert-435-error-message-erd -> origin/revert-435-error-message-erd
    ...
    
    601:  A Node.js module is loaded ('async_hooks' at line 6) which is not supported in the Edge Runtime.
    602:  Learn More: https://nextjs.org/docs/messages/node-module-in-edge-runtime
    603:  Import trace for requested module:
    604:  ../../../node_modules/.pnpm/[email protected]_@[email protected][email protected]_@[email protected]_@playwright+test@1._t3zvkidfsz7urkvhcheci25fnu/node_modules/flags/dist/chunk-OQHOV3IB.js
    605:  ../../../node_modules/.pnpm/[email protected]_@[email protected][email protected]_@[email protected]_@playwright+test@1._t3zvkidfsz7urkvhcheci25fnu/node_modules/flags/dist/next.js
    606:  ./libs/flags/index.ts
    607:  ./libs/index.ts
    608:  Linting and checking validity of types ...
    609:  Failed to compile.
    610:  ./src/functions/processSavePullRequest.ts:47:5
    611:  Type error: Argument of type 'bigint' is not assignable to parameter of type 'number'.
    612:  �[0m �[90m 45 |�[39m�[0m
    613:  �[0m �[90m 46 |�[39m   �[36mconst�[39m fileChanges �[33m=�[39m �[36mawait�[39m getPullRequestFiles(�[0m
    614:  �[0m�[31m�[1m>�[22m�[39m�[90m 47 |�[39m     repository�[33m.�[39minstallationId�[33m,�[39m�[0m
    615:  �[0m �[90m    |�[39m     �[31m�[1m^�[22m�[39m�[0m
    616:  �[0m �[90m 48 |�[39m     repository�[33m.�[39mowner�[33m,�[39m�[0m
    617:  �[0m �[90m 49 |�[39m     repository�[33m.�[39mname�[33m,�[39m�[0m
    618:  �[0m �[90m 50 |�[39m     payload�[33m.�[39mprNumber�[33m,�[39m�[0m
    619:  ELIFECYCLE  Command failed with exit code 1.
    620:  [ERROR] command finished with error: command (/home/runner/work/liam/liam/frontend/apps/app) /home/runner/setup-pnpm/node_modules/.bin/pnpm run build exited (1)
    ...
    
    652:  Generating static pages (0/31) ...
    653:  Generating static pages (7/31) 
    654:  Generating static pages (15/31) 
    655:  Generating static pages (23/31) 
    656:  ✓ Generating static pages (31/31)
    657:  Finalizing page optimization ...
    658:  Collecting build traces ...
    659:  ##[endgroup]
    660:  ##[error]@liam-hq/app#build: command (/home/runner/work/liam/liam/frontend/apps/app) /home/runner/setup-pnpm/node_modules/.bin/pnpm run build exited (1)
    661:  Tasks:    10 successful, 12 total
    662:  Cached:    0 cached, 12 total
    663:  Time:    2m9.497s 
    664:  Failed:    @liam-hq/app#build
    665:  ERROR  run failed: command  exited (1)
    666:  ELIFECYCLE  Command failed with exit code 1.
    667:  ##[error]Process completed with exit code 1.
    

    Copy link
    Contributor

    PR Reviewer Guide 🔍

    Here are some key observations to aid the review process:

    ⏱️ Estimated effort to review: 3 🔵🔵🔵⚪⚪
    🧪 No relevant tests
    🔒 No security concerns identified
    ⚡ Recommended focus areas for review

    Error Handling
    The webhook signature verification is called but the result is not checked. If signature verification fails, the code will continue execution instead of returning an error response.

    Type Mismatch

    The function returns file status types that don't match the expected types in GenerateReviewPayload. This could cause type errors when passing schemaChanges to the generateReviewTask.

    const schemaChanges = fileChanges.map((file) => {
      return {
        filename: file.filename,
        status: file.status,
        changes: file.changes,
        patch: file?.patch || '',
      }
    })
    Potential Error

    The generateReviewTask is triggered with schemaChanges that might contain status values not supported by the GenerateReviewPayload type, which only accepts 'added', 'modified', or 'deleted'.

    await generateReviewTask.trigger({
      ...payload,
      projectId: undefined,
      schemaChanges: result.schemaChanges,
    } as GenerateReviewPayload)

    Copy link
    Contributor

    PR Code Suggestions ✨

    Explore these optional code suggestions:

    CategorySuggestion                                                                                                                                    Impact
    Security
    Check signature verification result

    The webhook signature verification result is not being checked. The
    verifyWebhookSignature function likely returns a boolean that should be
    validated to ensure the request is authentic.

    frontend/apps/app/app/api/webhook/github/route.ts [13-14]

     validateConfig()
    -verifyWebhookSignature(await request.clone().text(), signature)
    +const isValid = verifyWebhookSignature(await request.clone().text(), signature)
    +if (!isValid) {
    +  return NextResponse.json({ error: 'Invalid signature' }, { status: 401 })
    +}
    • Apply this suggestion
    Suggestion importance[1-10]: 9

    __

    Why: This is a critical security issue. The PR removed the signature verification check that was present in the old code, which could allow unauthorized webhook requests to be processed. Adding this check is essential for maintaining security.

    High
    Possible issue
    Normalize status field values

    The status field in schemaChanges doesn't match the expected types in
    GenerateReviewPayload. The status in fileChanges has more possible values than
    the three ('added', 'modified', 'deleted') expected in the payload type.

    frontend/apps/app/src/functions/processSavePullRequest.ts [54-61]

     const schemaChanges = fileChanges.map((file) => {
    +  // Map GitHub status to expected status values
    +  let normalizedStatus: 'added' | 'modified' | 'deleted' = 'modified';
    +  if (file.status === 'added') normalizedStatus = 'added';
    +  else if (file.status === 'removed') normalizedStatus = 'deleted';
    +  
       return {
         filename: file.filename,
    -    status: file.status,
    +    status: normalizedStatus,
         changes: file.changes,
         patch: file?.patch || '',
       }
     })
    • Apply this suggestion
    Suggestion importance[1-10]: 8

    __

    Why: This suggestion addresses a type mismatch between the file status values from GitHub API and the expected values in GenerateReviewPayload. Without normalization, this could cause type errors or unexpected behavior when processing pull requests.

    Medium
    Fix unsafe type casting

    The type casting with as GenerateReviewPayload is unsafe because the spread
    operator with payload might not include all required properties of
    GenerateReviewPayload. This could lead to runtime errors if required properties
    are missing.

    frontend/apps/app/src/trigger/jobs.ts [30-34]

     await generateReviewTask.trigger({
    -  ...payload,
    +  pullRequestNumber: payload.pullRequestNumber,
       projectId: undefined,
    +  repositoryId: payload.repositoryId,
       schemaChanges: result.schemaChanges,
    -} as GenerateReviewPayload)
    +})
    • Apply this suggestion
    Suggestion importance[1-10]: 7

    __

    Why: The suggestion correctly identifies a potential type safety issue. Using explicit property assignment instead of spreading the payload ensures all required properties are properly included, preventing potential runtime errors.

    Medium
    • More

    Copy link
    Member

    @MH4GF MH4GF left a comment

    Choose a reason for hiding this comment

    The reason will be displayed to describe this comment to others. Learn more.

    Thanks!

    }

    const data = JSON.parse(payload) as GitHubWebhookPayload
    validateConfig()
    Copy link
    Member

    Choose a reason for hiding this comment

    The reason will be displayed to describe this comment to others. Learn more.

    This line is unnecessary, because Because it is done at server startup.

    ref: https://github.com/liam-hq/liam/blob/ignore-twice-build/frontend/apps/app/instrumentation.ts#L13

    Copy link
    Member Author

    Choose a reason for hiding this comment

    The reason will be displayed to describe this comment to others. Learn more.

    oh, good!

    Copy link
    Member Author

    @hoshinotsuyoshi hoshinotsuyoshi Mar 13, 2025

    Choose a reason for hiding this comment

    The reason will be displayed to describe this comment to others. Learn more.

    I'll fix that in this week 🙏

    @@ -6,6 +6,7 @@ import {
    import type { GitHubWebhookPayload } from '@/types/github'
    import { prisma } from '@liam-hq/db'

    // deprecated
    Copy link
    Member

    Choose a reason for hiding this comment

    The reason will be displayed to describe this comment to others. Learn more.

    👍🏻

    owner: payload.owner,
    name: payload.name,
    repositoryId: payload.repositoryId,
    } as SavePullRequestPayload)
    Copy link
    Member

    Choose a reason for hiding this comment

    The reason will be displayed to describe this comment to others. Learn more.

    Please do not use as if possible. Was it necessary?

    Copy link
    Member Author

    Choose a reason for hiding this comment

    The reason will be displayed to describe this comment to others. Learn more.

    Unnecessary. Applying #869 (comment) will solve that. 🙏

    Copy link
    Member Author

    Choose a reason for hiding this comment

    The reason will be displayed to describe this comment to others. Learn more.

    Regarding the conflict in this pull request and the lengthy Vercel deployment, I'll address these issues in a separate PR. 🙏

    Copy link
    Member

    Choose a reason for hiding this comment

    The reason will be displayed to describe this comment to others. Learn more.

    Looks good, Thank you 👍🏻

    Copy link
    Member

    @NoritakaIkeda NoritakaIkeda left a comment

    Choose a reason for hiding this comment

    The reason will be displayed to describe this comment to others. Learn more.

    I've reviewed the code and confirmed that it aligns with the intent of the PR. If there are no specific areas you'd like me to pay extra attention to, it's good to go! LGTM! 🚀

    Comment on lines +20 to +40
    try {
    const result = await processSavePullRequest({
    prNumber: payload.pullRequestNumber,
    owner: payload.owner,
    name: payload.name,
    repositoryId: payload.repositoryId,
    } as SavePullRequestPayload)
    logger.info('Successfully saved PR to database:', { prId: result.prId })

    // Trigger the next task in the chain - generate review
    await generateReviewTask.trigger({
    ...payload,
    projectId: undefined,
    schemaChanges: result.schemaChanges,
    } as GenerateReviewPayload)

    return result
    } catch (error) {
    logger.error('Error in savePullRequest task:', { error })
    throw error
    }
    Copy link
    Member

    Choose a reason for hiding this comment

    The reason will be displayed to describe this comment to others. Learn more.

    📝
    Save the pull request details to the database, then trigger the review generation.

    Comment on lines +35 to +46
    switch (action) {
    case 'opened':
    case 'synchronize':
    case 'reopened': {
    // Queue the savePullRequest task
    await savePullRequestTask.trigger({
    pullRequestNumber: pullRequest.number,
    projectId: undefined,
    owner: data.repository.owner.login,
    name: data.repository.name,
    repositoryId: data.repository.id,
    })
    Copy link
    Member

    Choose a reason for hiding this comment

    The reason will be displayed to describe this comment to others. Learn more.

    📝
    Trigger savePullRequestTask when a pull request is opened, synchronized, or reopened.

    Copy link
    Member

    Choose a reason for hiding this comment

    The reason will be displayed to describe this comment to others. Learn more.

    📝
    Check if the corresponding repository exists in the database, retrieve the PR's modified files, analyze the changed files, save them to the database, and return the PR data for the next process.

    @hoshinotsuyoshi hoshinotsuyoshi added this pull request to the merge queue Mar 13, 2025
    Merged via the queue into main with commit 1020c1b Mar 13, 2025
    20 checks passed
    @hoshinotsuyoshi hoshinotsuyoshi deleted the save-function-1 branch March 13, 2025 08:45
    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
    Projects
    None yet
    Development

    Successfully merging this pull request may close these issues.

    3 participants