Building files that can be interpreted by Vistumbler

Post Reply
arizonajon
Contributor
Contributor
Posts: 289
Joined: Wed Feb 04, 2015 11:17 pm
Contact:

Building files that can be interpreted by Vistumbler

Post by arizonajon »

Hey Andrew -

Just returned from another 60k AP trip to LA, and am getting enthusiastic about writing my own wardriving solution using a RPi, a GPS dongle, and a Wi-Fi dongle. It will be a lot more convenient than having to haul the pc around with me. I might even put it all in a box and mag mount it to the roof.

Looking at the .vs1 files, I see

|E0:10:7F:59:44:4C|Ruckus Wireless|WPA2-Enterprise|CCMP|3|802.11n|40|6,9,12,18,24,36,48,54||64|-68|Infrastructure|Unknown|1,64,-68\4,60,-70\7,62,-69\8,60,-70\10,60,-70\11,58,-71

for a typical AP capture. Most of the fields I think I understand, except the tuples at the end. I figure the "-xx" number is the "Signal Level", and the first number is the GPS entry index; is the middle number some kind of raw value from the Wi-Fi dongle?

When I run iwlist, here's an example of what I get:

Cell 17 - Address: 2C:7E:81:19:0D:EB
Channel:1
Frequency:2.412 GHz (Channel 1)
Quality=40/70 Signal level=-70 dBm
Encryption key:on
ESSID:"Apt 341"
Bit Rates:1 Mb/s; 2 Mb/s; 5.5 Mb/s; 11 Mb/s; 6 Mb/s
9 Mb/s; 12 Mb/s; 18 Mb/s
Bit Rates:24 Mb/s; 36 Mb/s; 48 Mb/s; 54 Mb/s
Mode:Master
Extra:tsf=0000000000000000
Extra: Last beacon: 10ms ago
IE: Unknown: 000741707420333431
IE: Unknown: 010882848B968C129824
IE: Unknown: 030101
IE: Unknown: 050400010000
IE: Unknown: 0706555320010B1E
IE: Unknown: 2A0100
IE: Unknown: 3204B048606C
IE: Unknown: 460573D000000C
IE: Unknown: 2D1AAD011BFFFFFF00000000000000000001000000000406E6470D00
IE: Unknown: 3D1601000500000000000000000000000000000000000000
IE: Unknown: 7F0800000F0200000040
IE: Unknown: DD180050F2020101840003A4000027A4000042435E0062322F00
IE: Unknown: DD0900037F01010000FF7F
IE: Unknown: DD1D0050F204104A0001101044000102103C0001021049000600372A000120
IE: IEEE 802.11i/WPA2 Version 1
Group Cipher : CCMP
Pairwise Ciphers (1) : CCMP
Authentication Suites (1) : PSK

So, for this AP, it seems that I'd record the current GPS location as index location 1, in this format
1|N 3356.7375|W 11823.1883|09|1.34|78.74|-33.00|35.47|22.03|271.30|2019-08-22|14:45:03.116 (CR)(LF)

then for the Wi-Fi AP entry, I can see the MAC, the SSID, the RSSI in dBm, the quality of the signal (is that related to the middle number of your tuple?). So for the above entry, is it |1,40,-70 ? How to I get manufacturer, or "WPA2-Enterprise", etc? Is that a look-up table in Vistumbler? Or, can I just leave those fields blank? Basically, provide an entry that looks like this:

|2C:7E:81:19:0D:EB|NA|WPA2 Version 1|CCMP|NA|RadioType|1| 6,9,12,18,24,36,48,54||40|-70|SomethingforInfrastructure|1,40,-70(CR)(LF)

If the fields that I've munged up in some way should actually contain info that comes from the iwlist output, which are which? And should that signal level, since the iwlist output says "40/70", be something more like int(100*(40/70)) = 57, if it's to be normalized to a 0-100 scale?

Cheers and 73 - Jon N7UV
arizonajon
Contributor
Contributor
Posts: 289
Joined: Wed Feb 04, 2015 11:17 pm
Contact:

Re: Building files that can be interpreted by Vistumbler

Post by arizonajon »

and the point is, I'd like the output files from my app to be readable by the WiFiDB app, and maybe even importable via the uploader tool.
arizonajon
Contributor
Contributor
Posts: 289
Joined: Wed Feb 04, 2015 11:17 pm
Contact:

Re: Building files that can be interpreted by Vistumbler

Post by arizonajon »

or, if I somehow get Kismet up and running, does that produce files that can be read by WiFiDB?
User avatar
ACalcutt
Vistumbler / TechIdiots Admin
Vistumbler / TechIdiots Admin
Posts: 1302
Joined: Sun Oct 21, 2007 6:50 pm
Location: Rutland, MA
Contact:

Re: Building files that can be interpreted by Vistumbler

Post by ACalcutt »

Jon,

Right now WifiDB doesn't support kismet, but if you can get me a proper test file I can see what I can do to get it supported. I've never had a good example with gps information included, but if they have some type of text/xml output I can look to support it.


For building your own app and making a vs1, It's a good test, but try not to lose information by forcing it into the vs1 format. In the wifidb, the original approach to other formats was to convert them to vs1, but I found some of those other formats had additional information that was good to have but was getting lost in the conversion. where now I am taking the approach of trying to standardize the data, but keeping the unique bits of each format

There are a few things you may want check out to help get some of the missing values you mention.

This is the current VS1 format information: https://github.com/acalcutt/Vistumbler/ ... VS1-Format , it tells about things like what the signal pairs mean (GPS ID / SIGNAL / RSSI values)

For some of your tougher questions, I think some examples functions from the wifidb would help.
(In this file https://github.com/acalcutt/WifiDB/blob ... rt.inc.php)
check out these functions
Sig2dBm (line 32)
dBm2Sig (line 43)
findCapabilities (line 146)
findFreq (line 212)

Sig2dBm and dBm2Sig convert back and forth between the signal and rssi dbm value. I found this formula somewhere years ago and it seemed to be the most accurate i found for converting between the two. It expects MaxSignal of -30 DissociationSignal of -85 . (note this is just my best guess based on testing a lot of different conversion methods many years ago. a direct linear conversion didn't seem to work)

findCapabilities takes the flags most programs gives and converts them into the vistumbler (really netsh) format Authentication/Encryption. Your output seems to have all the pieces to convert it to a matching vistumbler format, but it doesn't quite seem to be just straight flags (the parts are spread over the bottom 4 lines, so you would have to find the right way to put them together)

findFreq will take a frequency and change it to a channel. it looks like your program provides both, so you may not need this one, but a few of the formats I support only have frequency, and I want the channel in the wifidb since i is used as one of the items that makes an AP unique


Also to note, I always found detailed CSV, which I made after the VS1 was easier to work with, mainly because it doesn't separate AP and GPS data so you don't have to match GIDs. (https://github.com/acalcutt/Vistumbler/ ... CSV-Format).
arizonajon
Contributor
Contributor
Posts: 289
Joined: Wed Feb 04, 2015 11:17 pm
Contact:

Kismet output files

Post by arizonajon »

Hey Andrew -

I finally got kismet running on an RPi. Took most of the day, mainly due to *nix learning curve issues.

What I have learned is that kismet doesn't have a single output type that handily has the usual stuff that Vistumbler provides. For instance, there's 5 files in the log I ran just now using an old 2.4 GHz dongle hanging by the window.

Kismet-20190824-21-32-23-1.alert
Kismet-20190824-21-32-23-1.gpsxml
Kismet-20190824-21-32-23-1.nettxt
Kismet-20190824-21-32-23-1.netxml
Kismet-20190824-21-32-23-1.pcapdump

Running kismet for just two minutes created files that range from 0 to 1.4 MiB. pcampdump is the biggest, I can't open it with a text editor, but I'd expect it to have all the raw packets recorded. The .netxml and .nettxt files are essentially the same content, one formatted more for meatbags, the other for computers. The .gpsxml has individual rows like this:
<gps-point bssid="9C:3D:CF:98:2F:EB" source="9C:3D:CF:98:2F:EB" time-sec="1566707545" time-usec="91640" lat="33.492477" lon="-112.073006" spd="0.085000" heading="0.000000" fix="3" alt="347.299988" signal_dbm="-71" noise_dbm="0"/>

But, there's no SSID in this file, so one needs to go to the other files that have both ssid and MAC.

The .netxml file appears to have two main parts: the header, then the data. The data is listed by detected SSID networks, then the clients that are communicating with that SSID.

So here's the header:

<?xml version="1.0" encoding="ISO-8859-1"?>
<!DOCTYPE detection-run SYSTEM "http://kismetwireless.net/kismet-3.1.0.dtd">

<detection-run kismet-version="2016.07.R1" start-time="Sat Aug 24 21:32:23 2019">

<card-source uuid="5531be8a-c6f1-11e9-80e9-dd04bc23e301">
<card-source>wlan1</card-source>
<card-name>wlan1</card-name>
<card-interface>wlan1</card-interface>
<card-type>rt2800usb</card-type>
<card-packets>5585</card-packets>
<card-hop>true</card-hop>
<card-channels>1,5,9,2,6,10,3,7,11,4,8</card-channels>
</card-source>

and the first of many APs detected:

<wireless-network number="1" type="infrastructure" first-time="Sat Aug 24 21:32:26 2019" last-time="Sat Aug 24 21:34:19 2019">
<SSID first-time="Sat Aug 24 21:32:26 2019" last-time="Sat Aug 24 21:34:19 2019">
<type>Beacon</type>
<max-rate>216.700000</max-rate>
<packets>79</packets>
<beaconrate>10</beaconrate>
<wps>No</wps>
<encryption>WPA+TKIP</encryption>
<encryption>WPA+AES-CCM</encryption>
<wpa-version>WPA+WPA2</wpa-version>
<essid cloaked="true"></essid>
</SSID>
<BSSID>02:A0:0D:11:CF:16</BSSID>
<manuf>Unknown</manuf>
<channel>11</channel>
<freqmhz>2462 79</freqmhz>
<maxseenrate>1000</maxseenrate>
<packets>
<LLC>79</LLC>
<data>0</data>
<crypt>0</crypt>
<total>79</total>
<fragments>0</fragments>
<retries>0</retries>
</packets>
<datasize>0</datasize>
<snr-info>
<last_signal_dbm>-67</last_signal_dbm>
<last_noise_dbm>0</last_noise_dbm>
<last_signal_rssi>0</last_signal_rssi>
<last_noise_rssi>0</last_noise_rssi>
<min_signal_dbm>-75</min_signal_dbm>
<min_noise_dbm>0</min_noise_dbm>
<min_signal_rssi>1024</min_signal_rssi>
<min_noise_rssi>1024</min_noise_rssi>
<max_signal_dbm>-65</max_signal_dbm>
<max_noise_dbm>-256</max_noise_dbm>
<max_signal_rssi>0</max_signal_rssi>
<max_noise_rssi>0</max_noise_rssi>
</snr-info>
<gps-info>
<min-lat>33.492458</min-lat>
<min-lon>-112.073029</min-lon>
<min-alt>342.000000</min-alt>
<min-spd>0.007000</min-spd>
<max-lat>33.492489</max-lat>
<max-lon>-112.072983</max-lon>
<max-alt>347.500000</max-alt>
<max-spd>0.393000</max-spd>
<peak-lat>33.492477</peak-lat>
<peak-lon>-112.072998</peak-lon>
<peak-alt>347.299988</peak-alt>
<avg-lat>33.492475</avg-lat>
<avg-lon>-112.073009</avg-lon>
<avg-alt>345.089800</avg-alt>
</gps-info>
<bsstimestamp>Jul 4 02:23:41</bsstimestamp>
<cdp-device></cdp-device>
<cdp-portid></cdp-portid>
<seen-card>
<seen-uuid>5531be8a-c6f1-11e9-80e9-dd04bc23e301</seen-uuid>
<seen-time>Sat Aug 24 21:34:19 2019</seen-time>
<seen-packets>79</seen-packets>
</seen-card>

Then a client connected to that SSID:
<wireless-client number="1" type="fromds" first-time="Sat Aug 24 21:32:26 2019" last-time="Sat Aug 24 21:34:19 2019">
<client-mac>02:A0:0D:11:CF:16</client-mac>
<client-manuf>Unknown</client-manuf>
<channel>11</channel>
<freqmhz>2462 79</freqmhz>
<maxseenrate>1000</maxseenrate>
<packets>
<LLC>79</LLC>
<data>0</data>
<crypt>0</crypt>
<total>79</total>
<fragments>0</fragments>
<retries>0</retries>
</packets>
<datasize>0</datasize>
<snr-info>
<last_signal_dbm>-67</last_signal_dbm>
<last_noise_dbm>0</last_noise_dbm>
<last_signal_rssi>0</last_signal_rssi>
<last_noise_rssi>0</last_noise_rssi>
<min_signal_dbm>-75</min_signal_dbm>
<min_noise_dbm>0</min_noise_dbm>
<min_signal_rssi>1024</min_signal_rssi>
<min_noise_rssi>1024</min_noise_rssi>
<max_signal_dbm>-65</max_signal_dbm>
<max_noise_dbm>-256</max_noise_dbm>
<max_signal_rssi>0</max_signal_rssi>
<max_noise_rssi>0</max_noise_rssi>
</snr-info>
<gps-info>
<min-lat>33.492458</min-lat>
<min-lon>-112.073029</min-lon>
<min-alt>342.000000</min-alt>
<min-spd>0.007000</min-spd>
<max-lat>33.492489</max-lat>
<max-lon>-112.072983</max-lon>
<max-alt>347.500000</max-alt>
<max-spd>0.393000</max-spd>
<peak-lat>33.492477</peak-lat>
<peak-lon>-112.072998</peak-lon>
<peak-alt>347.299988</peak-alt>
<avg-lat>33.492475</avg-lat>
<avg-lon>-112.073009</avg-lon>
<avg-alt>345.089800</avg-alt>
</gps-info>
<seen-card>
<seen-uuid>5531be8a-c6f1-11e9-80e9-dd04bc23e301</seen-uuid>
<seen-time>Sat Aug 24 21:34:19 2019</seen-time>
<seen-packets>79</seen-packets>
</seen-card>
</wireless-client>

So, wow, there's a ton of stuff in this one file. And it's 1.2 MiB in size, after only 2 minutes. There's SSID, a bunch of info about the current location, average location, etc., a trillion ways to look at signal strength, and so many other things. But I don't see a 1-1 relationship between signal strength and location in this file. For my own AP, it shows my 6 currently connected clients.

In the .nettxt file, it's more simple (well, sorta). Here's the same first network.

Network 1: BSSID 02:A0:0D:11:CF:16
Manuf : Unknown
First : Sat Aug 24 21:32:26 2019
Last : Sat Aug 24 21:34:19 2019
Type : infrastructure
BSSID : 02:A0:0D:11:CF:16
SSID 1
Type : Beacon
SSID : "" (Cloaked)
First : Sat Aug 24 21:32:26 2019
Last : Sat Aug 24 21:34:19 2019
Max Rate : 216.7
Beacon : 10
Packets : 79
WPS : No
Encryption : WPA+TKIP
Encryption : WPA+AES-CCM
WPA Version: WPA+WPA2
Channel : 11
Frequency : 2462 - 79 packets, 100.00%
Max Seen : 1000
LLC : 79
Data : 0
Crypt : 0
Fragments : 0
Retries : 0
Total : 79
Datasize : 0
Min Pos : Lat 33.492458 Lon -112.073029 Alt 342.000000 Spd 0.007000
Max Pos : Lat 33.492489 Lon -112.072983 Alt 347.500000 Spd 0.393000
Peak Pos : Lat 33.492477 Lon -112.072998 Alt 347.299988
Avg Pos : AvgLat 33.492475 AvgLon -112.073009 AvgAlt 345.089800
Last BSSTS : Jul 4 02:23:41
Seen By : wlan1 (wlan1) 5531be8a-c6f1-11e9-80e9-dd04bc23e301 79 packets
Sat Aug 24 21:34:19 2019
Client 1: MAC 02:A0:0D:11:CF:16
Manuf : Unknown
First : Sat Aug 24 21:32:26 2019
Last : Sat Aug 24 21:34:19 2019
Type : From Distribution
MAC : 02:A0:0D:11:CF:16
Channel : 11
Frequency : 2462 - 79 packets, 100.00%
Max Seen : 1000
LLC : 79
Data : 0
Crypt : 0
Fragments : 0
Retries : 0
Total : 79
Datasize : 0
Min Pos : Lat 33.492458 Lon -112.073029 Alt 342.000000 Spd 0.007000
Max Pos : Lat 33.492489 Lon -112.072983 Alt 347.500000 Spd 0.393000
Peak Pos : Lat 33.492477 Lon -112.072998 Alt 347.299988
Avg Pos : AvgLat 33.492475 AvgLon -112.073009 AvgAlt 345.089800
Seen By : wlan1 (wlan1) 5531be8a-c6f1-11e9-80e9-dd04bc23e301 79 packets
Sat Aug 24 21:34:19 2019

Here, I can see all the necessary bits that would align with what the WiFiDB currently shows, but still, the gps location is a min/max/avg kind of thing.

So, it appears to me that elements of both the .gpsxml and .netxml files need to be correlated first to extract the usual WiFiDB info.

I will send you the files if you're interested.
User avatar
ACalcutt
Vistumbler / TechIdiots Admin
Vistumbler / TechIdiots Admin
Posts: 1302
Joined: Sun Oct 21, 2007 6:50 pm
Location: Rutland, MA
Contact:

Re: Building files that can be interpreted by Vistumbler

Post by ACalcutt »

the nettxt or netxml woudl probably be the way to go. the xml one would probably be easier to iterate through on my end. seems like it has everything needed...but I would only consider this one point, probably <max_signal_dbm>-65</max_signal_dbm> and <peak-lat>33.492477</peak-lat><peak-lon>-112.072998</peak-lon>
arizonajon
Contributor
Contributor
Posts: 289
Joined: Wed Feb 04, 2015 11:17 pm
Contact:

Re: Building files that can be interpreted by Vistumbler

Post by arizonajon »

here are both of those files from my 2 minute run. i added a .txtsuffix to each to get by the uploader. Just remove that and you'll have them as they were.
Attachments
Kismet-20190824-21-32-23-1.netxml.txt
(323.84 KiB) Downloaded 721 times
Kismet-20190824-21-32-23-1.nettxt.txt
(147.45 KiB) Downloaded 774 times
arizonajon
Contributor
Contributor
Posts: 289
Joined: Wed Feb 04, 2015 11:17 pm
Contact:

Re: Building files that can be interpreted by Vistumbler

Post by arizonajon »

I did a short (~1 mi) drive just now, attached are the .netxml and .nettxt files. Man, they grow fast. I need to figure out a way to throttle the output a bit.
Attachments
Kismet-20190825-08-30-04-1.netxml.txt
(4.24 MiB) Downloaded 739 times
Kismet-20190825-08-30-04-1.nettxt.txt
(1.9 MiB) Downloaded 786 times
arizonajon
Contributor
Contributor
Posts: 289
Joined: Wed Feb 04, 2015 11:17 pm
Contact:

Re: Building files that can be interpreted by Vistumbler

Post by arizonajon »

you'll note that I don't have the manufacturer's database installed. I tried to do it by following the procedure, but got errors each time.
User avatar
ACalcutt
Vistumbler / TechIdiots Admin
Vistumbler / TechIdiots Admin
Posts: 1302
Joined: Sun Oct 21, 2007 6:50 pm
Location: Rutland, MA
Contact:

Re: Building files that can be interpreted by Vistumbler

Post by ACalcutt »

I made a basic importer for the netxml. right now you can use it by selecting "Kismet netkml" on the import page.

It seemed to at least work with the test files you gave me ( https://wifidb.net/wifidb/opt/userstats ... arizonajon and https://wifidb.net/wifidb/opt/userstats ... arizonajon)

I am still wondering what a EAP/WPA-Enterprise AP looks like in a kismet export to make sure i got it right.

For the manufacturers not being in the file it really doesn't matter. WifiDB uses its own manufacturer database.

For cutting down the file size, is there any way to take out the "wireless-client" sections or does it make you export them?
User avatar
ACalcutt
Vistumbler / TechIdiots Admin
Vistumbler / TechIdiots Admin
Posts: 1302
Joined: Sun Oct 21, 2007 6:50 pm
Location: Rutland, MA
Contact:

Re: Building files that can be interpreted by Vistumbler

Post by ACalcutt »

also, is the wifi adapter in your kismet machine only b/g? just curious because it the file lists the aps as "802.11b+"
arizonajon
Contributor
Contributor
Posts: 289
Joined: Wed Feb 04, 2015 11:17 pm
Contact:

Re: Building files that can be interpreted by Vistumbler

Post by arizonajon »

Yep, it was an old one that Raspbian auto-recognizes. The newer (like 4 y old one) with dual band and (maybe) ac needs to have the driver compiled, and I haven't done that yet.
User avatar
ACalcutt
Vistumbler / TechIdiots Admin
Vistumbler / TechIdiots Admin
Posts: 1302
Joined: Sun Oct 21, 2007 6:50 pm
Location: Rutland, MA
Contact:

Re: Building files that can be interpreted by Vistumbler

Post by ACalcutt »

I added a new release of the WifiDB Uploader that can also upload netxml files. https://github.com/acalcutt/WiFiDBClien ... s/tag/v1.3

I was thinking it's probably not to hard to make a bash script that can do the file upload from linux
arizonajon
Contributor
Contributor
Posts: 289
Joined: Wed Feb 04, 2015 11:17 pm
Contact:

Re: Building files that can be interpreted by Vistumbler

Post by arizonajon »

hey - this is cool. I will give it a try soon.

WarPig1.0 is the name of my RPi with kismet. Wrote it up here at my blog jonadams.com/wireless/?p=1027

This past week I spent time learning about vpn. I have a number of remote RPis or Windows boxes, all collecting telemetry or listening to radio channels, and at least for the Windows boxes have been using Teamviewer to connect and configure them. Two things I don't like about this, one is that I'd rather not run Windows boxes as they consume too much power and are pretty finicky, and second Teamviewer, while serviceable, has been dunning me again with "COMMERCIAL USE SUSPECTED" notices, which I'm tired of.

Since most of what I want to do remotely can be done on an RPi, I thought it'd be good to learn to set up a vpn connection between these remote devices and home and be able to interact with them directly. All of my remote stuff is behind firewalls (like someone's home router/firewall, or a cellular modem with firewall), but ultimately what I'd like to have is a suite of connectivity functions on an RPi that allows that remote Pi to phone home, set up a vpn connection for control and monitoring, and use the local internet connection to fire the data (like audio streams, or data streams) to other web-based servers (like broadcastify, or whatever). Don't need that high-bandwidth data coming through the vpn pipe and then using my home connection to get to the internet.

So, I've gotten openvpn set up on two RPis, one as the server within my local subnet, and the other as a client. I connected the client to my phone's hotspot (not my local subnet) and boom! within moments I had an inbound vpn connection from the RPi via AT&T to my network. Pretty darned cool, but I'm easily impressed.

I can ping both ways, the vpn subnet is a 10 network, and my home network is a 192, but the pings route just fine. I can file transfer using scp, configure remote USB sound cards, it seems to work pretty well. I have not yet figured out how to do the "split tunnel" on the client, where communications between my local subnet and the client go over the VPN, but the audio streams from the client route out over their local Internet connection. I'm sure there's a way to do it, it probably involves iptables, but haven't figured it out. Do you have any insight on that?

Back to warpig1.0, I still have to compile and install a driver for the dual-band wi-fi dongle that I have available. Raspbian doesn't recognize it right now. Or, maybe it's time to purchase a somewhat newer one and perhaps that will solve the issue if it has better support in *nix.

Thanks for all your work! Cheers and 73 - Jon N7UV
arizonajon
Contributor
Contributor
Posts: 289
Joined: Wed Feb 04, 2015 11:17 pm
Contact:

Re: Building files that can be interpreted by Vistumbler

Post by arizonajon »

Can I install two Wi-Fi adapters on the mobile PC running Vistumbler and Windows 7? I'd like to improve my scan rate, and was thinking I could set one to 2.4 GHz and the other to 5 GHz. Maybe even more than two, if that works. Right now the only way I can envision this working is by running two instances of Vistumbler, one for each dongle. Or, is there a trick that makes this work?

Cheers and 73 - Jon N7UV
User avatar
ACalcutt
Vistumbler / TechIdiots Admin
Vistumbler / TechIdiots Admin
Posts: 1302
Joined: Sun Oct 21, 2007 6:50 pm
Location: Rutland, MA
Contact:

Re: Building files that can be interpreted by Vistumbler

Post by ACalcutt »

The easiest way is to use portable mode, which goes to the old way of storing the settings file right inside the vistumbler directory instead off %AppData%

There are two ways to get portable mode in the latest release.

1.) The "ZIP Portable" Release on the homepage starts in portable mode, so you just extract it to a folder and its ready to go. If you extract to muliple folders you can have multiple vistumbler setups using different settings

2.) I also added Options -> Portable mode switch in the last release, which switches the installer version to portable mode (you may need to run vistumbler as administrator if you installed to Program Files though, otherwise it may fail to write the setting file due to permissions)
User avatar
ACalcutt
Vistumbler / TechIdiots Admin
Vistumbler / TechIdiots Admin
Posts: 1302
Joined: Sun Oct 21, 2007 6:50 pm
Location: Rutland, MA
Contact:

Re: Building files that can be interpreted by Vistumbler

Post by ACalcutt »

Personally if I was attempting what your trying to do with the RasPis, I don't think I would use vpn. I would probably just stick to either SSH with a pre shared key, or just plain HTTPS with a cert (letsencrypt is free and easy with their autobots). I think either of those would be plenty secure to transfer the data home. Open a SSH/HTTPS port on a home server that all the pis can communicate with, set up a nice scheduled data sync (either by flat file or into a database). Since the pis would just be sending home, you wouldn't really need a connection into them (unless you feel they need management, then that is another story)
User avatar
ACalcutt
Vistumbler / TechIdiots Admin
Vistumbler / TechIdiots Admin
Posts: 1302
Joined: Sun Oct 21, 2007 6:50 pm
Location: Rutland, MA
Contact:

Re: Building files that can be interpreted by Vistumbler

Post by ACalcutt »

For warpig, maybe just grab a Pi Zero W. It at least has an b/g/n adapter built in and it looks easy to mod for an external antenna. I've gotten a few for small projects and they work pretty well.

https://www.raspberrypi.org/products/ra ... pi-zero-w/
https://www.briandorey.com/post/raspber ... ntenna-mod

EDIT: thinking about this further, with the pi zero w, there is no wired connections, so it wouldn't be able to used both the wifi for lan and scanning (unless perhaps you got a usb lan card...its probably better to use a better higher power external wifi adapter anyway though)
arizonajon
Contributor
Contributor
Posts: 289
Joined: Wed Feb 04, 2015 11:17 pm
Contact:

Re: Building files that can be interpreted by Vistumbler

Post by arizonajon »

ACalcutt wrote: Tue Sep 10, 2019 5:09 am ... Since the pis would just be sending home, you wouldn't really need a connection into them (unless you feel they need management, then that is another story)
Hey Andrew - thanks for the feedback. I'm pretty sure it is "another story". The remote Pi generates a data stream that I don't need to receive at the house. That stream goes to a 3rd party server somewhere else, and I don't want to deal with the two-way relay of that data through my home firewall and limited upstream service.

But, I do need to be able to control and monitor the remote Pi, which is why I need a secure way of doing that and having that link always phone home. It'd be kinda like I had a remote Vistumbler device along a busy highway, where I would ideally want the data stream to just supply directly to your WiFiDB server, but I'd want to be able to control the device from my location.

Now, to your point, the remote pi doesn't need to be always connected to home for control/monitoring, but if it phoned home say every 5 minutes, I could intercept one of those inbound calls and establish the two-way control/monitoring session.

My reading so far suggests that the only way to do what I need is to have the remote Pi establish a split tunnel, where comms with the home subnet go through the VPN tunnel, and comms to the outside world go direct. AFAIK, that means that the home vpn server has a set of route rules, when the remote pi connects, the routing rule set is forwarded to the pi, and the pi does its thing.

I did set up a vpn server here using an old pi, it seems to work fine, but all traffic comes home. So maybe soon I'll learn how the split tunnel needs to be set up and solve that bit. In addition, I need to have a firewall for the remote pi, so that the part exposed to the internet is protected.

If you still think there's an easier path than the one that I'm traveling, i'm all ears.

cheers and 73 - Jon N7UV
arizonajon
Contributor
Contributor
Posts: 289
Joined: Wed Feb 04, 2015 11:17 pm
Contact:

Re: Building files that can be interpreted by Vistumbler

Post by arizonajon »

I ordered and received this week an Alfa AWUS036AC dual-band USB Wi-Fi dongle. I am going to connect it to warpig1.0 and see what a kismet run looks like with it. I haven't dug into kistmet logging yet to see what flexibility it might have to restrict what is logged, but I'm sure that there's something that can be done to reduce the log file size.
arizonajon
Contributor
Contributor
Posts: 289
Joined: Wed Feb 04, 2015 11:17 pm
Contact:

Re: Building files that can be interpreted by Vistumbler

Post by arizonajon »

Happy T-Day, Andrew -

I tried installing and running the Uuloader on a "new" truck PC running Win10. When I attempt to start it, WIndows intercepts it and says it's running a security scan, but the scan never finishes or something else occurs, and in any event the uploader doesn't want to run. I've tried running it as admin and that didn't make any difference either. I'm really new to Win10 (yeah, where have I been...) but am trying to find a way to prevent Windows from getting in the way of running the uploader, in case that's why it never starts up.
I'll work on it again tomorrow, but thought I'd just ask in case there was something obvious that I didn't do.
Cheers and 73 - Jon N7UV
arizonajon
Contributor
Contributor
Posts: 289
Joined: Wed Feb 04, 2015 11:17 pm
Contact:

Re: Building files that can be interpreted by Vistumbler

Post by arizonajon »

Hey Andrew -

Saw that you were around, thought I'd revive my issue here. Ever since I upgraded my mobile computer to Win10P, I cannot get Uploader to run. I double-click on it, the icon looks like something's about to happen, then nada. Nothing obvious in the Task Monitor, either. So that's the first issue.

Second one is that because of the first, I've tried bringing all my log files to the home pc (venerable W7P box) where Uploader was generally consistently working. However, there, as soon as I start it, it does the countdown exit even though there are hundreds of new files.

So adjunct to the second question, how does Uploader do its thing? If the first hash in date-ordered files is claimed to be already uploaded, is that when it stops? Or does it move through the whole directory to see if there's a new file?

Cheers and 73 - Jon N7UV
User avatar
ACalcutt
Vistumbler / TechIdiots Admin
Vistumbler / TechIdiots Admin
Posts: 1302
Joined: Sun Oct 21, 2007 6:50 pm
Location: Rutland, MA
Contact:

Re: Building files that can be interpreted by Vistumbler

Post by ACalcutt »

Which version of the uploader are you trying to use? Version 1.3 here? https://github.com/acalcutt/WiFiDBClient/releases . Version 1.3 expects at least .Net 4.5.2

For the one can you can open, can you hit the X on the countdown, then go check your Settings -> Auto Settings and make sure the folder is correct and a place you can open.? Also, if it looks right you can turn on Debug and see in the logs what is happening, though it is sometimes hard to follow because it is multiple threads mixed together is some cases.
User avatar
ACalcutt
Vistumbler / TechIdiots Admin
Vistumbler / TechIdiots Admin
Posts: 1302
Joined: Sun Oct 21, 2007 6:50 pm
Location: Rutland, MA
Contact:

Re: Building files that can be interpreted by Vistumbler

Post by ACalcutt »

Jon, can you give this v1.4 version a try. It expects at least version 4.6.2 of .net, which was needed to properly fix an SSL related error i saw in testing that looked to be causing it not to work,

https://github.com/acalcutt/WiFiDBClien ... s/tag/v1.4
arizonajon
Contributor
Contributor
Posts: 289
Joined: Wed Feb 04, 2015 11:17 pm
Contact:

Re: Building files that can be interpreted by Vistumbler

Post by arizonajon »

Well, maybe it's my bad. I upgraded to V1.4 and it seems to be working now... You should be receiving about 2200 files over the next hour or so %^)

Cheers - Jon N7UV
arizonajon
Contributor
Contributor
Posts: 289
Joined: Wed Feb 04, 2015 11:17 pm
Contact:

Re: Building files that can be interpreted by Vistumbler

Post by arizonajon »

as I was uploading, I got this several times:
Capture.PNG
Capture.PNG (226.14 KiB) Viewed 28802 times
arizonajon
Contributor
Contributor
Posts: 289
Joined: Wed Feb 04, 2015 11:17 pm
Contact:

Re: Building files that can be interpreted by Vistumbler

Post by arizonajon »

Here's the details of another unhandled exception:
See the end of this message for details on invoking
just-in-time (JIT) debugging instead of this dialog box.

************** Exception Text **************
System.IO.FileNotFoundException: Could not find file 'G:\Vistumbler_fEB2020\2020-02-21 16-41-40_AutoSave.VS1'.
File name: 'G:\Vistumbler_fEB2020\2020-02-21 16-41-40_AutoSave.VS1'
at System.IO.__Error.WinIOError(Int32 errorCode, String maybeFullPath)
at System.IO.FileInfo.get_Length()
at WiFiDBUploader.WiFiDBUploadMainForm.InsertNewListViewRow(String[] split, String Type)
at WiFiDBUploader.WiFiDBUploadMainForm.backgroundWorker_ImportProgressChanged(Object sender, ProgressChangedEventArgs e)
at System.ComponentModel.BackgroundWorker.OnProgressChanged(ProgressChangedEventArgs e)
at System.ComponentModel.BackgroundWorker.ProgressReporter(Object arg)


************** Loaded Assemblies **************
mscorlib
Assembly Version: 4.0.0.0
Win32 Version: 4.8.4069.0 built by: NET48REL1LAST_B
CodeBase: file:///C:/Windows/Microsoft.NET/Framework/v4.0.30319/mscorlib.dll
----------------------------------------
WDB_Uploader
Assembly Version: 1.2.0.0
Win32 Version: 1.2.0.0
CodeBase: file:///C:/Users/jta/Downloads/WDB_Uploader_v1.4%20(1)/WDB_Uploader_v1.4/WDB_Uploader.exe
----------------------------------------
System.Windows.Forms
Assembly Version: 4.0.0.0
Win32 Version: 4.8.4042.0 built by: NET48REL1LAST_C
CodeBase: file:///C:/Windows/Microsoft.Net/assembly/GAC_MSIL/System.Windows.Forms/v4.0_4.0.0.0__b77a5c561934e089/System.Windows.Forms.dll
----------------------------------------
System
Assembly Version: 4.0.0.0
Win32 Version: 4.8.4001.0 built by: NET48REL1LAST_C
CodeBase: file:///C:/Windows/Microsoft.Net/assembly/GAC_MSIL/System/v4.0_4.0.0.0__b77a5c561934e089/System.dll
----------------------------------------
System.Drawing
Assembly Version: 4.0.0.0
Win32 Version: 4.8.3761.0 built by: NET48REL1
CodeBase: file:///C:/Windows/Microsoft.Net/assembly/GAC_MSIL/System.Drawing/v4.0_4.0.0.0__b03f5f7f11d50a3a/System.Drawing.dll
----------------------------------------
WDBTraceLog
Assembly Version: 1.0.0.0
Win32 Version: 1.0.0.0
CodeBase: file:///C:/Users/jta/Downloads/WDB_Uploader_v1.4%20(1)/WDB_Uploader_v1.4/WDBTraceLog.DLL
----------------------------------------
System.Configuration
Assembly Version: 4.0.0.0
Win32 Version: 4.8.3761.0 built by: NET48REL1
CodeBase: file:///C:/Windows/Microsoft.Net/assembly/GAC_MSIL/System.Configuration/v4.0_4.0.0.0__b03f5f7f11d50a3a/System.Configuration.dll
----------------------------------------
System.Core
Assembly Version: 4.0.0.0
Win32 Version: 4.8.4110.0 built by: NET48REL1LAST_B
CodeBase: file:///C:/Windows/Microsoft.Net/assembly/GAC_MSIL/System.Core/v4.0_4.0.0.0__b77a5c561934e089/System.Core.dll
----------------------------------------
System.Xml
Assembly Version: 4.0.0.0
Win32 Version: 4.8.3761.0 built by: NET48REL1
CodeBase: file:///C:/Windows/Microsoft.Net/assembly/GAC_MSIL/System.Xml/v4.0_4.0.0.0__b77a5c561934e089/System.Xml.dll
----------------------------------------
WDBSQLite
Assembly Version: 1.0.0.0
Win32 Version: 1.0.0.0
CodeBase: file:///C:/Users/jta/Downloads/WDB_Uploader_v1.4%20(1)/WDB_Uploader_v1.4/WDBSQLite.DLL
----------------------------------------
WDBAPI
Assembly Version: 1.0.0.0
Win32 Version: 1.0.0.0
CodeBase: file:///C:/Users/jta/Downloads/WDB_Uploader_v1.4%20(1)/WDB_Uploader_v1.4/WDBAPI.DLL
----------------------------------------
WDBCommon
Assembly Version: 1.0.0.0
Win32 Version: 1.0.0.0
CodeBase: file:///C:/Users/jta/Downloads/WDB_Uploader_v1.4%20(1)/WDB_Uploader_v1.4/WDBCommon.DLL
----------------------------------------
System.Data.SQLite
Assembly Version: 1.0.99.0
Win32 Version: 1.0.99.0
CodeBase: file:///C:/Users/jta/Downloads/WDB_Uploader_v1.4%20(1)/WDB_Uploader_v1.4/System.Data.SQLite.DLL
----------------------------------------
System.Data
Assembly Version: 4.0.0.0
Win32 Version: 4.8.3761.0 built by: NET48REL1
CodeBase: file:///C:/Windows/Microsoft.Net/assembly/GAC_32/System.Data/v4.0_4.0.0.0__b77a5c561934e089/System.Data.dll
----------------------------------------
System.Transactions
Assembly Version: 4.0.0.0
Win32 Version: 4.8.3761.0 built by: NET48REL1
CodeBase: file:///C:/Windows/Microsoft.Net/assembly/GAC_32/System.Transactions/v4.0_4.0.0.0__b77a5c561934e089/System.Transactions.dll
----------------------------------------
System.EnterpriseServices
Assembly Version: 4.0.0.0
Win32 Version: 4.8.3761.0 built by: NET48REL1
CodeBase: file:///C:/Windows/Microsoft.Net/assembly/GAC_32/System.EnterpriseServices/v4.0_4.0.0.0__b03f5f7f11d50a3a/System.EnterpriseServices.dll
----------------------------------------
System.Xml.Linq
Assembly Version: 4.0.0.0
Win32 Version: 4.8.3761.0 built by: NET48REL1
CodeBase: file:///C:/Windows/Microsoft.Net/assembly/GAC_MSIL/System.Xml.Linq/v4.0_4.0.0.0__b77a5c561934e089/System.Xml.Linq.dll
----------------------------------------

************** JIT Debugging **************
To enable just-in-time (JIT) debugging, the .config file for this
application or computer (machine.config) must have the
jitDebugging value set in the system.windows.forms section.
The application must also be compiled with debugging
enabled.

For example:

<configuration>
<system.windows.forms jitDebugging="true" />
</configuration>

When JIT debugging is enabled, any unhandled exception
will be sent to the JIT debugger registered on the computer
rather than be handled by this dialog box.
User avatar
ACalcutt
Vistumbler / TechIdiots Admin
Vistumbler / TechIdiots Admin
Posts: 1302
Joined: Sun Oct 21, 2007 6:50 pm
Location: Rutland, MA
Contact:

Re: Building files that can be interpreted by Vistumbler

Post by ACalcutt »

this might have been me messing with the network today, but it looks like this was just before I started. from there error it looks like it was having trouble reading the file on your side. "Could not find file 'G:\Vistumbler_fEB2020\2020-02-21 16-41-40_AutoSave.VS1'.'"

Maybe could you have two copies of the uploader open and the auto archive enabled?
Post Reply