Exports missing some images #34
Comments
|
We probably need to inspect individual images post-warping, and console log out their lat/lon coords too. |
|
More examples in #23 |
|
😕 |
|
#32 (comment) maybe found the issue, or part of it! |
|
Let's solve this first - then go back to cropping:
we solved the flipped image. Now let's get all images showing. |
|
Logs from that last run: |
|
The only image showing is this one: |
|
Hmm. I really can't see why the 2nd and 3rd images aren't getting composited over the first. Maybe I should try uploading the 2nd and 3rd to cloud storage so we can inspect them... |
|
I'm adding a new route |
|
We should be able to view with http://export.mapknitter.org/working/1558570858/w306187.tif although the cluster may mess things up... we'll see. |
|
Alternatively we can upload our working files... |
|
But, we'd have to modify the gem to do that, since the gem doesn't. Maybe we can find the filenames brute force with a |
|
Otherwise I'll try to run it in development mode on Tycho maybe. |
|
OK, interesting. i was able to download the working images, and all three look good; here are JPG conversions of them:
|
|
And using
That was without setting the bounds in the first step. But when I did this: I got our bug! Great!
So, what's wrong with our first step:
|
|
OK, i don't know what was wrong, but a combined command fixes it, and I've released a new gem v1.0.5, which uses a combined command, like: Fingers 🤞 this should solve the issue! |
|
We did it!!!!
|
|
This may now be solved, as well as #32 -- we should test on a large map! |
|
http://export.mapknitter.org/id/1558588105/status.json (update: this got to 36/44 and stalled) http://export.mapknitter.org/id/1558588271/status.json (update: this got to 36/44 and stalled) I accidentally ran 2 at once, and now both seem stalled? It's a really big export... and maybe the scale is too big... but not sure, one got to 1 of 44, and the other got to 3 of 44, and now both seem stuck. Could a redeploy have happened, interrupting them? |
|
Ok restarted at scale 50, /id/1558617025/status.json Also noticed an export running late in the night towards 44 got to 3x... |
|
Did it stall after that? |
|
1558617025 seems to have gotten to 44 images! So, in the logs: Then i see a /lot/ of About 20m later, with lots of Then it continues:
Aha! It then saved |
|
OK, yeah - core dumped! Found it, earlier in the process; it was the 3rd image in the sequence: So, when this happens, should we try to get it to retry with a |
|
Like: begin
retries ||= 0
# try to warp
rescue Exception => e
puts "failed to warp #{filename}"
retry if (retries += 1) < 3
end |
|
LOL ohhhhhhhh |
|
This was from https://mapknitter.org/maps/irish-uk-border-mapping/ |
|
Haha, got it. There was one image that was stretched wildly out of proportion, and went into the Irish sea by hundreds of miles:
We should think about how to catch these! |
|
I deleted it and am trying again: http://export.mapknitter.org/export?url=https://mapknitter.org/maps/irish-uk-border-mapping/warpables.json&scale=50 |
|
Accidentally started 2 at once... we'll see how it goes... |
|
OK, this is still running... |
|
|
|
OK, it got through warping and to compositing. But another image is missing:
I'll see if it's wildly out of shape. Hmm. It doesn't look bad. I'll see if there was a memory dump. |
|
OK! It's a filename encoding error: Hmm, how to fix this? |
|
Testing on another set to try to fix the |
|
Dealing with the |
|
And, that big map worked - update at #23 |
Not sure where this is happening, it could be in
mapknitter-exporter:This map: https://mapknitter.org/maps/ceres--2/
Looks like this on MapKnitter:
Output in this app is:
The text was updated successfully, but these errors were encountered: