Completely lost with migration

[Martii]

So our project is currently using the old github dependency and we've been trying to migrate to the latest usage for the GitHub API v3... however...

There are a bazillion questions... here are some things done...

1. We realize the QSP are no longer going to be used by May 5th, 2021 ... known this for a long time.
2. Which packages, and I do mean plural, are the starting point? I have https://www.npmjs.com/package/@octokit/auth-oauth-app#usage-with-octokit which can authenticate our app using our existing id and secret but then I go to what used to be github dep and then it says use https://www.npmjs.com/package/@octokit/rest which then it references migration to 14 is as easy as changing the require name which of course doesn't work. However the auth-oauth-app says use https://www.npmjs.com/package/@octokit/core ... which ones?
3. https://www.npmjs.com/package/@octokit/auth-token allegedly gets the access_token needed to test if https://www.npmjs.com/package/@octokit/auth-oauth-app#usage-with-octokit ... so not sure what to do with this yet.
4. I've read the simple guide at https://docs.github.com/en/rest/guides/getting-started-with-the-rest-api which helps using curl but that's not node.js

My main question is where the heck do I start with what packages for migration? i.e. obviously we want the larger rate limit (we'll skip PAT for now and use our app OAuth) but there doesn't seem to be any clear, and current, documentation on migration.

Please help.

P.S. I've also searched the web and got a lot of outdated articles or way too general. I've also reached out to the original contributor and of course he's gone dark which isn't helpful.

[wolfy1339]

You start out with @octokit/rest, and you can add on authentication as such:

const { Octokit } = require("@octokit/rest");
const { createAppAuth } = require("@octokit/auth-app");

const appOctokit = new Octokit({
  authStrategy: createAppAuth,
  auth: {
    appId: 123,
    privateKey: process.env.PRIVATE_KEY,
    // optional: this will make appOctokit authenticate as app (JWT)
    //           or installation (access token), depending on the request URL
    installationId: 123,
  },
});

@octokit/app-auth combines the different ways of authenticating as an app for the GitHub API. It does JWT and OAuth.

You can use any other auth plugin the same way, instead of using @octokit/core as is listed in their documentation, you use @octokit/rest

Here are the docs for @octokit/rest regarding authentication: https://octokit.github.io/rest.js/v18#authentication

[Martii]

I did look into Personal Access Tokens (PAT) and eventually want to go down that road but I seem to recall over a year ago they can use the same type? Would the combined one (the first one) be better for that?

[wolfy1339]

@octokit/app-auth is for GitHub apps.

I don't think you want Personal Access Tokens. Those are for individuals and not apps.

[wolfy1339]

Maybe @gr2m can give more insight here

[Martii]

My thoughts were if our accounts that have GitHub as a strategy were to have their own rate/limiting maximums by using their own authorizations it would increase what they could do for scalability of the site.

Right now we're pretty small so OAuth only (with our limits) could do for now and is what we currently use with the old github package. For some reason the original contributor starts our app out with the github authorization... I commented that out and then all the sudden I couldn't log into the site (the callback hung from GH)... also may have hit the "brown out" period earlier too but have to run tests on migrations for a few days so that should avoid those periods. There's another one scheduled later today so won't be twiddling at those times. 😸

[Martii]

Hmmm... I tried using a test https://www.npmjs.com/package/@octokit/rest with https://github.com/octokit/auth-oauth-app.js and it throws a rejection saying that I need to use https://www.npmjs.com/package/@octokit/auth-oauth-user

(node:12815) UnhandledPromiseRejectionWarning: Error: [@octokit/auth-oauth-app] "GET /users/{username}/repos" does not support clientId/clientSecret basic authentication. Use @octokit/auth-oauth-user instead.

With our sites clientId and clientSecret is when this happens when I list my own repos with await test.rest.repos.listForUser({ username: 'Martii' }); If I run unauthenticated it works fine... but limited to 60 per hour.

---

If I use https://www.npmjs.com/package/@octokit/auth-oauth-user then the initialization of the Octokit constructor says I need a code which I'm pretty sure we don't have those for any users. All we have is the strategy auth value and that doesn't seem to work (says bad or expired code). EDIT https://github.com/octokit/auth-oauth-app.js#oauth-web-flow says it was the code sent in the callback when authorized... pretty sure we don't save that.

[gr2m]

tl;dr - try this

const { Octokit } = require("@octokit/core");
const { paginateRest } = require("@octokit/plugin-paginate-rest");

const MyOctokit = Octokit.plugin(paginateRest).defaults({ userAgent: "my-app-name" });
const oauthToken = Buffer.from([process.env.CLIENT_ID, process.env.CLIENT_SECRET].join(':')).toString('base64')

const octokit = new MyOctokit({ auth: `basic ${oauthToken}` });

octokit.paginate("GET /users/{username}/repos", { username: "Martii" }).then(repositories => {
  console.log("%d repositories found", repositories.length)
})

So our project is currently using the old github dependency

😱

Wow, that's been ... a while

However the auth-oauth-app says use https://www.npmjs.com/package/@octokit/core ... which ones?

The authentication strategies such as @octokit/auth-oauth-app require @octokit/core. @octokit/rest is build upon @octokit/core since v16. It's only a few lines of code now: https://github.com/octokit/rest.js/blob/master/src/index.ts

There were a ton of changes to REST API the methods since the github package.

What I would suggest is to migrate to @octokit/core and not use the endpoint methods at all, instead use octokit.request():
https://github.com/octokit/core.js#rest-api-example

The benefit is that the REST API endpoints barely change. You get full TypeScript support, too, the same as using the REST API methods:

It looks like you use endpoints that paginate, in that case I would recommend to use the paginate plugin:
https://github.com/octokit/plugin-paginate-rest.js

https://www.npmjs.com/package/@octokit/auth-token allegedly gets the access_token needed to test if https://www.npmjs.com/package/@octokit/auth-oauth-app#usage-with-octokit ... so not sure what to do with this yet.

@octokit/auth-token is built into @octokit/core. Just set the auth option to a token, as shown here:
https://github.com/octokit/core.js#authentication

I tried using a test https://www.npmjs.com/package/@octokit/rest with https://github.com/octokit/auth-oauth-app.js and it throws a rejection saying that I need to use https://www.npmjs.com/package/@octokit/auth-oauth-user

(node:12815) UnhandledPromiseRejectionWarning: Error: [@octokit/auth-oauth-app] "GET /users/{username}/repos" does not support clientId/clientSecret basic authentication. Use @octokit/auth-oauth-user instead.

I think this is actually a bug. When using with an OAuth App, it should set the apps clientId/clientSecret as basic authentication in order to bump the rate limit. I'll create a follow up issue to look into that. I've recently split up @octokit/auth-oauth-app into @octokit/auth-oauth-app and @octokit/auth-oauth-user, this bug might have been introduced as a result. Sorry for that.

For the time being, can you try this

const oauthToken = Buffer.from([process.env.CLIENT_ID, process.env.CLIENT_SECRET].join(':')).toString('base64')
const octokit = new Octokit({
  auth: `basic ${oauthToken}`,
});

That should work, I didn't test it though.

If I use https://www.npmjs.com/package/@octokit/auth-oauth-user then the initialization of the Octokit constructor says I need a code which I'm pretty sure we don't have those for any users

This is the code you get from the OAuth web flow, as you found out. This code expires quickly, there is no use in storing it, it's meant to be exchanged immediately for an OAuth user access token, which is what @octokit/auth-oauth-user does for you as soon as you send the first request.

[Martii]

Wow, that's been ... a while

That's because this dependency has been quite volatile and when I see the semver major going up like a slot machine on crack I usually avoid updating... but I'm here now, by force (May the 5th be with you too) 🙄

There were a ton of changes to REST API the methods since the github package.

Yah but migration, with constant change, is the way of JavaScript... everyone is never happy with something that works well already. 😸

It's only a few lines of code now: https://github.com/octokit/rest.js/blob/master/src/index.ts

Well I'm kind of needing the request logger portion of @octokit/rest... just found it in the github package and it was nice to verify that requests to the API were indeed authenticated vs when they were not. Not entirely sure what the legacy REST endpoint ones are in @octokit/rest but I'm sure I'll find out. So I'm leaning towards @octokit/rest rather than learning about the plugin initialization. Plus it already has the pagination plugin built in. I'll just assume @octokit/rest is equivalent to a meta package until told differently.

this bug might have been introduced as a result. Sorry for that.

Bugs happen. 😸 No problem... we'll wait.

can you try this

const oauthToken = Buffer.from([process.env.CLIENT_ID, process.env.CLIENT_SECRET].join(':')).toString('base64')
const octokit = new Octokit({
  auth: `basic ${oauthToken}`,
});

Thanks and will do... but need slumber first so I don't pull all my remaining hair out in one shot... we have something similar but didn't find out until moments ago that the code that was modified was unreachable.

but whilst I'm away if you get a moment please... I've been hunting for an alternate construction of Octokit... we don't use environment variables and we utilize a DB lookup for credentials. So every example I've seen, including here, uses environment variables. We won't be doing that... so we really need a way to create the Octokit instance, then authorize it after a DB lookup and then send it on it's module exports way.

Since I haven't seen anything even remotely like this anywhere... how is this done please? ❓ Link that I haven't found in the last few years? ❓

---

I do see the .ts comments at https://github.com/octokit/core.js/blob/2bcc013/src/index.ts#L121-L125 but it's still a bit greek to me atm.

1.) Looks familiar with https://github.com/octokit/auth-oauth-app.js with the 'unauthenticated' bit. i.e. passing no object in the constructor
2.) Would seem like it's what @wolfy1339 did up there. Is that correct? ❓
3.) Sounds like our dead code that I found earlier. Is this saying that I need to provide my own request package management with authentication or not? ❓
TODO.) Is this what you asked me to try out up there? ❓

[Martii]

Verbatim Test

CODE:

const { Octokit } = require("@octokit/core");
const { paginateRest } = require("@octokit/plugin-paginate-rest");

const MyOctokit = Octokit.plugin(paginateRest).defaults({ userAgent: "my-app-name" });
const oauthToken = Buffer.from([process.env.CLIENT_ID, process.env.CLIENT_SECRET].join(':')).toString('base64')

const octokit = new MyOctokit({ auth: `basic ${oauthToken}` });

octokit.paginate("GET /users/{username}/repos", { username: "Martii" }).then(repositories => {
  console.log("%d repositories found", repositories.length)
})

RESULT:

(async function () {
^

TypeError: octokit.paginate(...).then(...) is not a function

COMMENT(S):
Uhmmmm? Aroo? ❓

Modified test

CODE: (We're partially limited to ES5.1 btw but syntactically the same... that's another issue atm)

var Octokit = require("@octokit/rest").Octokit;

var clientId = 'Can not post this publicly but it is accurate';
var clientSecret = 'Can not post this publicly but it is accurate';

var oauthToken = Buffer.from([`${clientId}`, `${clientSecret}`].join(':')).toString('base64');

var test = new Octokit({
  auth: `basic ${oauthToken}`
});

test.paginate("GET /users/{username}/repos", { username: "Martii" }).then(repositories => {
  console.log("%d repositories found", repositories.length)
});

RESULT:

...
(node:48411) UnhandledPromiseRejectionWarning: HttpError: Bad credentials
    at ~/repo/git/OpenUserJS.org/martii/OpenUserJS.org/node_modules/@octokit/request/dist-node/index.js:68:23
    at processTicksAndRejections (internal/process/task_queues.js:93:5)
    at async Object.next (~/repo/git/OpenUserJS.org/martii/OpenUserJS.org/node_modules/@octokit/plugin-paginate-rest/dist-node/index.js:62:26)
...

COMMENT(S):
Our credentials are absolutely correct and pulled directly from the DB and plugged in as strings directly for this test. Authenticates with a limit of 5000/hour with the github dependency and verified with the debug: true on init. Also verified with https://www.npmjs.com/package/@octokit/auth-oauth-user and its property that is returned for authorization. i.e. they match for oauthToken for the hash.

---

Ref(s):
package.json snippet

    "@octokit/auth-oauth-app": "4.1.2",
    "@octokit/auth-oauth-user": "1.2.4",
    "@octokit/rest": "18.5.2",

    "@octokit/core": "3.4.0",
    "@octokit/plugin-paginate-rest": "2.13.3"

$ node -v
v14.16.1

---

Btw unrelated question... what are these here in Discussions? no tool tip and its very vague on what to do with the arrow click:

EDIT Saw a tooltip when I was logged out... it's a vote up for the solution button.

---

Restested in package.json snippet with newly released:

"@octokit/rest": "18.5.3",

... same issue of "Bad credentials" as encountered in 18.5.2.

[Martii]

@gr2m

Is octokit/auth-oauth-app.js#178 related to the failures experienced in your verbatim code and the modified test?

[gr2m]

yeah, that bug should be resolved now. When using @octokit/auth-oauth-app you can now send requests to public resources and the authorization header will be set to basic auth from client ID/secret