The website uploader has a number of issues

Post Reply
arizonajon
Contributor
Contributor
Posts: 289
Joined: Wed Feb 04, 2015 11:17 pm
Contact:

The website uploader has a number of issues

Post by arizonajon »

Hi Andrew -

First, thanks again for continuing to support our AP data collection. It's a bad habit I have...

I have not been able to get WiFiDB Uploader to do anything but open, I enter the upload name, then point it to the right folder, then it immediately says "closing in 30 seconds...". This has been going on for months now.

So, I have quite a backlog. Today, I chose to try the website's Import File page instead of the Uploader. That seems to work ok, but even there I cannot upload more than about 30 files at a time. I get the following error:

"Failure... File not supplied for . Try one of the supported file types."

So I dial it back to 25, then if that gives the same error, 20, then 15, etc. Eventually, it does the Import properly, and no single file seems to trigger that alarm. It's just slower since the batch size is so much smaller.

Using this method, I've tried importing the same file more than once, and it appears to properly ignore the dupe. That I expect as well.

Just now, I tried to import a number of files at a time, and got a new error:
12345Capture.PNG
12345Capture.PNG (15.33 KiB) Viewed 18555 times
ANy ideas or advice? I'd really like to go back to using the Uploader, but in the mean time I'll use what works, albeit sometimes fitfully.

Cheers and 73 - Jon N7UV
User avatar
ACalcutt
Vistumbler / TechIdiots Admin
Vistumbler / TechIdiots Admin
Posts: 1302
Joined: Sun Oct 21, 2007 6:50 pm
Location: Rutland, MA
Contact:

Re: The website uploader has a number of issues

Post by ACalcutt »

For the app can you try this. When it starts counting down the 30 seconds, close the countdown window so it cancels. Then manually go to File -> Exit. Once closed, try reopening it and see if it actually uploads. I've noticed an issue where the database is not deleting with the autoclose, so it doesn't behave properly on the next load.

Also, If your are not on release v1.4 of the uploader, please upgrade it because there was a TLS related issue in previous versions ( https://github.com/acalcutt/WiFiDBClient/releases/ )


For the upload through the website, is it actually letting you upload multiple files? I didn't think that form supported more than one file at a time....or at least...it wasn't made to

The error in your screenshot seems to indicate more than one file with the same hash was uploaded (the same file was uploaded twice before the uploader could process it). I'll have to see how I can improve that.
arizonajon
Contributor
Contributor
Posts: 289
Joined: Wed Feb 04, 2015 11:17 pm
Contact:

Re: The website uploader has a number of issues

Post by arizonajon »

Good morning Andrew -

I am on 1.4. Maybe I will delete it and re-install it. Yeah, I do kill the countdown and then try it manually, but doesn't seem to begin a process. I'll watch it with Task Manager.

The website does seem to allow multiple files per submit. My record is about 31. However, I do not know what parameter causes it to break. Last night before I quit for the day I was investigating whether there was a maximum aggregate size to the upload. I've got more files to go, so maybe I can sort that out.

Cheers and 73 - Jon N7UV
User avatar
ACalcutt
Vistumbler / TechIdiots Admin
Vistumbler / TechIdiots Admin
Posts: 1302
Joined: Sun Oct 21, 2007 6:50 pm
Location: Rutland, MA
Contact:

Re: The website uploader has a number of issues

Post by ACalcutt »

Good to see you by the way. The uploads are dead without you...it ends up being mostly me.

I've been doing a lot of working with the map. Right now I have been trying to generate a TerrainRGB file using whole planet images from the Japanese space agency (jaxa aw3d30). Supposedly this would allow me to have 3d terrain in the maps and hillshade using the maplibre library that is coming soon. I have already used the jaxa images to make my own color shading and hillshading, but supposedly with the terrainrgb I can do it directly in the library, plus do some extra 3d stuff. we will see

Unfortunately trying to make the map files uses a lot of resources. I just build a second server which has been giving me a bit of trouble. I've had it go down on my 3 times now and I've had my websites running on there...so thats no good. Still trying to find the issue there...it seemed to be working better after rearing thing, but i seemed to have a random restart last night.

I finally got a file to generate today, but it took 165GB of memory and almost 500GB swap on an nvme disk. It is basically taking 23,942 25MB image files, projecting them in a virtual raster, and wring them into a mbtiles file and various zoom levels.
User avatar
ACalcutt
Vistumbler / TechIdiots Admin
Vistumbler / TechIdiots Admin
Posts: 1302
Joined: Sun Oct 21, 2007 6:50 pm
Location: Rutland, MA
Contact:

Re: The website uploader has a number of issues

Post by ACalcutt »

I've made one change that should at least the multiple hash warning you got above. it might help a little. Basically made it so the same hash can be uploaded twice, at least until the import process gets to it and sees it is a duplicate
arizonajon
Contributor
Contributor
Posts: 289
Joined: Wed Feb 04, 2015 11:17 pm
Contact:

Re: The website uploader has a number of issues

Post by arizonajon »

Thanks - glad to continue contributing!

I've been collecting mostly non-stop, but the uploader hadn't been performing for me, so i kinda vanished. Am am finally attempting to get the truck cleaned up again, with new radios, and some mechanical repairs that were needed.

I do have a recurrent problem with Vistumbler that I haven't diagnosed sufficiently. I have a USB GPS unit, on COM2. When I start up the PC, and then Vistumbler, Vistumbler finds the GPS stream and seems to work per spec. If I then park the truck, and leave it there for a *while* (15-30 min) when I come back and start moving again Vistumbler seems to be stuck on the last few GPS positions from before I parked it. And it's several, like it's jittering in GPS position by 10,000's of a degree. The GPS button shows green, but does not read new positions. If I check the GPS output independently, it's updating positions properly. It's almost like there's a buffer in Vistumbler that is choked with the last 5-10 positions and never clears. If I close and reopen Vistumbler the problem goes away.

Does any of that sound familiar?

Cheers and 73 -Jon N7UV
User avatar
ACalcutt
Vistumbler / TechIdiots Admin
Vistumbler / TechIdiots Admin
Posts: 1302
Joined: Sun Oct 21, 2007 6:50 pm
Location: Rutland, MA
Contact:

Re: The website uploader has a number of issues

Post by ACalcutt »

Does the computer go to sleep when it sits for a while? My guess it would loose connection to the gps if it did, but then think it was still connected.

Also, if i remember correctly... I though gps was more accurate when moving. when sitting still it tends to wander I thought.
User avatar
ACalcutt
Vistumbler / TechIdiots Admin
Vistumbler / TechIdiots Admin
Posts: 1302
Joined: Sun Oct 21, 2007 6:50 pm
Location: Rutland, MA
Contact:

Re: The website uploader has a number of issues

Post by ACalcutt »

I was thinking about you mentioning that the gps position would get stuck alternating the last few gps positions. Technically vistumbler itself does not keep old gps positions, it only keeps the latest. If it is alternating between different positions, it is at least reading the gps buffer or making it appear that way.
Post Reply