Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Download comment history #134

Open
AlexAplin opened this issue May 14, 2023 · 1 comment
Open

Download comment history #134

AlexAplin opened this issue May 14, 2023 · 1 comment

Comments

@AlexAplin
Copy link
Owner

Nico uniquely has a rolling comment list with the most recent comments replacing past comments. However, users can filter the comment list by date to view comments from that point in time. That's reflected in requests made to https://nvcomment.nicovideo.jp/v1/threads containing:

"additionals":{"when":1210714200}

We should offer a flag to specify a timestamp (either epoch or %Y-%m-%d %H:%s) and/or to fetch all comments between two timestamps and aggregate them to one file.

@AlexAplin
Copy link
Owner Author

AlexAplin commented May 14, 2023

Requesting by epoch means there is tremendous granularity here, and you'll only receive 250 of the comments up to that timestamp, ordered ascending. Unfortunately the actual comments don't come with their own epoch time, but we do get a total comment count per thread, and each comment has a timestamp and is numbered overall:

commentCount: 30079

postedAt: '2007-09-22T08:42:17+09:00'
no: 29826

So broadly I think what needs to happen is we request the newest comments as normal with no when set. Then, take the first comment postedAt, convert to epoch, and request at that point (may need to add some buffer here to ensure we don't miss anything). Collect based on no to prevent duplicates and to identify when we've reasonably gotten all comments.

We probably will need to refresh our thread key periodically, which is done with OPTIONS and GET to https://nvapi.nicovideo.jp/v1/comment/keys/thread?videoId=. So far we haven't done this.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

1 participant