Time interval of records in detailed CSV export

[ulusoyca]

Hello, I am using Vistumbler to export the RSSI values in CSV format and I am reading these values in MATLAB. I have got a question and I would be gratefu if you can answer me. When I export the AP history in detailed CSV, I see the record times not continously lets say log per 1 second for each AP.

I just want to know how Vistumbler CSV export works. I mean sometimes there is long time difference between two records of one AP and sometimes it is less than one second.

"aalto open",00:27:0D:08:C3:20,"Cisco Systems",36,Open,None,802.11g,006,1 2 5.5 11,6 9 12 18 24 36 48 54,Infrastructure,"Unknown",0.0000000,0.0000000,00,0,0,0,0,0,0,2012-08-15,11:29:50.288
"aalto open",00:27:0D:08:C3:20,"Cisco Systems",43,Open,None,802.11g,006,1 2 5.5 11,6 9 12 18 24 36 48 54,Infrastructure,"Unknown",0.0000000,0.0000000,00,0,0,0,0,0,0,2012-08-15,11:30:17.569
"aalto open",00:27:0D:08:C3:20,"Cisco Systems",43,Open,None,802.11g,006,1 2 5.5 11,6 9 12 18 24 36 48 54,Infrastructure,"Unknown",0.0000000,0.0000000,00,0,0,0,0,0,0,2012-08-15,11:30:18.583
"aalto open",00:27:0D:08:C3:20,"Cisco Systems",43,Open,None,802.11g,006,1 2 5.5 11,6 9 12 18 24 36 48 54,Infrastructure,"Unknown",0.0000000,0.0000000,00,0,0,0,0,0,0,2012-08-15,11:30:19.590
"aalto open",00:27:0D:08:C3:20,"Cisco Systems",43,Open,None,802.11g,006,1 2 5.5 11,6 9 12 18 24 36 48 54,Infrastructure,"Unknown",0.0000000,0.0000000,00,0,0,0,0,0,0,2012-08-15,11:30:20.597

In short, how is the time interval between last two records chosen?

Regards,

[ACalcutt]

There are two things that affect the interval of vistumbler. One is controllable by the user, the other is not.

The controllable thing is the "Refresh loop time" setting (Settings --> Vistumbler Settings --> Refresh loop time). By default this is set to 1000ms(1s). This is basically a rate limiter and prevents vistumbler from scanning faster than the value specified (with it set at 1000ms it will scan APs at about once per second)

The uncontrollable thing is how AutoIT works. AutoIT, which vistumbler is written in, is a scripting language. It runs the code very linearly in a single process, so if something takes long in a single part of code, it can delay the rest of the code from running, causing a gap like you are seeing.

If you look at the top of vistumbler (next to the scan/gps buttons) you will see some text that says "Actual loop time". This value is basically how long the last it took to scan, add to the gui, do all the extra functions. At minimum it should be around whatever you have "Refresh loop time" set to, but things like getting access points, writing them back to the gui, etc can make this time increase. If you watch "Actual loop time", this is basically the gaps between the items in your csv file.


All that said the main thing that usually slow vistumbler down
- Adding a large amount of access points to gui list. List actions in autoit seem slow, if there are a lots of them it makes the script take longer (and in my script increasing the "actual loop time')
- Using the option "View -> Add new APs to top". In my code, adding to the top of the gui list take longer because it have to move all the other list items down in the process
- Filters where things are constantly being added or removed (another listview thing)
- Any extended function, like the speak signal, auto wifidb, KML can add a delay, though I usually do my best to try and prevent that (like AutoKML uses a seperate process)
- Doing anything in the GUI. Things like moving, resizing, scrolling, going into menus in the gui will cause the script to pause, which can add some gaps in time

[ulusoyca]

Thanks for the explanation. It helps me to understand the CSV file.

Another question: What is Auto Refresh Networks? WHat will change if I enable it?

I would like to give feedbacks about Vistumbler and its benefits with my work. I want to measure the signal strength to see the Wifi interference with Bluetooth. So I am using Vistumbler to see the fluctations of WiFi strength during the measurement. Hence, I need samples at each second. If there is a gap I am assuming that it is the same value with most recent value.

1- Detailed CSV format is very useful. Using this file I can generate my own timeline graphics such as:



2- CSV format is sometimes tricky since some APs contains "," comma in their names and it makes confusion when I import data.

3- It would be great if there was an option to filter the APs whose strength is higher than a threshold signal strength level. It might increase the performance. Because I would like to observe the 5 strongest APs.

[ACalcutt]

What is Auto Refresh Networks? WHat will change if I enable it?

Auto Refresh Networks forces a scan of access points on an interval (set in Settings --> Vistumbler Settings --> Auto Refresh networks). With this disabled you would notice the signal doesn't update as often (since it updates at whatever rate windows refreshes instead of at the set interval). I recommend keep this enabled unless you are using XP or Vista (this function causes some slowness issues in xp and vista)

CSV format is sometimes tricky since some APs contains "," comma in their names and it makes confusion when I import data.

The chance of commas is why AP names are surrounded by quotes in the csv files. You could always import into something like excel (which won't have a problem with AP SSID commas due to the quotes) and export a tab delimited file instead

It would be great if there was an option to filter the APs whose strength is higher than a threshold signal strength level. It might increase the performance. Because I would like to observe the 5 strongest APs.

There is a way to do this if the APs are active.
1.)Create a Filter (View --> Filters --> Add/Remove Filter -> Add Filter) and set the Signal field to a range (say you want 50% - 100% to show, you use 50-100)
2.)Set the filter to the one you created above View --> Filters --> (The filter name you create)
3.)Export Filtered APs to CSV (File --> Export --> Export to CSV --> Filtered APs). This will only export APs that meet the filter criteria (the ones showing in the GUI)


If you are using vistumbler to view interference you may want to check out the latest beta (v10.3 Beta 18, only available through automatic update right now) which added support for RSSI in dBm. The RSSI dBm value seems to be much more sensitive (especially in the 99% signal area) and may be better for your graphs. To get the latest beta make sure "Automatically Check For Updates" and "Check For Beta Updates" are checked in (Settings --> Vistumbler Settings)

[ulusoyca]

If you are using vistumbler to view interference you may want to check out the latest beta (v10.3 Beta 18, only available through automatic update right now) which added support for RSSI in dBm. The RSSI dBm value seems to be much more sensitive (especially in the 99% signal area) and may be better for your graphs. To get the latest beta make sure "Automatically Check For Updates" and "Check For Beta Updates" are checked in (Settings --> Vistumbler Settings)

Thank you for the answer again. Converting from RSSI to dbm is something dependent on Vendor of wireless card. There are different formulations for Cisco, Atheros and Intel cards. I think Vistumbler uses one formulation as mentioned in one of your posts. Is the estimation close enough?

[ACalcutt]

Thank you for the answer again. Converting from RSSI to dbm is something dependent on Vendor of wireless card. There are different formulations for Cisco, Atheros and Intel cards. I think Vistumbler uses one formulation as mentioned in one of your posts. Is the estimation close enough?

In the latest beta if you use native wifi mode (Options --> Use Native Wifi) the RSSI value is pulled directly from the card. RSSI is not estimated like previous versions of vistumber (if you don't switch to native wifi mode it will still be an estimated value). Native wifi mode has been greatly improved in the beta.

[ulusoyca]

That sounds very promising since I spent days to find the formulation for Intel cards and I couldnt find.

Can you write here the download page of the latest beta version of Vistumbler? I couldnt find it on the website. When is its release date?

Regards

[ACalcutt]

Like I said above, right now its only available though vistumblers automatic update. To get the latest beta make sure "Automatically Check For Updates" and "Check For Beta Updates" are checked in (Settings --> Vistumbler Settings).

I would say a final release of 10.3 is close. I basically have everything done that I wanted in the beta.

[ulusoyca]

I should thank you very much with your help. Now I got exactly what I wanted with the Beta version.



And I wanted to share my MATLAB code that shows the dBm of the strongest 3 APs at each second. What you should do is just deleting the first line of your "detailed CSV" exported files which contains the title of each columns.

Good luck!

Code: Select all

%% Read File
clear all
clc
filename='test.csv';
fid = fopen(filename);
xls_file = strcat(filename,'.xls');
C=textscan(fid, '%q %s %q %s %s %f %s %s %s %s %s %q %q %s %q %s %s %s %s %s %s %s %s %s %s %s','delimiter', ',');
fclose(fid);
%% Variables
Signal=C{6}; % Signal strength: RSSI_percentage 
AP_mac=C{2}; % MAC addresses of APs
Date=C{25};
Time=C{26};

len=length(Signal); % Number of records
Num_of_AP=1; % Number of APs

len_Date=length(Date);
date=cell(len_Date,1);
time=cell(len_Date,1);
Seconds=cell(len_Date,1);
datetime=cell(len_Date,1);

for i=1:len_Date
time{i} = textscan(Time{i}, '%f %f %f', 'delimiter', ':');
x=i;
end

offset=time{1}{1}*3600+time{1}{2}*60+time{1}{3}-1;

for i=1:len_Date
Seconds{i} = round(time{i}{1}*3600+time{i}{2}*60+time{i}{3}-offset);
end



%% Finds how many APs there are; 
for i=1:len-1
 if strcmp(AP_mac(i),AP_mac(i+1)) == 0
     Num_of_AP=Num_of_AP+1;
 end
end

%% Finds the quantity of samples per AP
AP_list=cell(1,Num_of_AP);
index=1; % index of each AP
count=0; % AP number
for j=1:len-1
 if strcmp(AP_mac(j),AP_mac(j+1)) == 0
     %RSSI=(RSSI_MAX/100)*Signal(j); %Convert to real RSSI 
     AP_list{index,count+1}= Signal(j); % RSSI-112; dbm
     count=count+1;
     index=1;
 else
     %RSSI=(RSSI_MAX/100)*Signal(j); %Convert to real RSSI 
     AP_list{index,count+1}= Signal(j);% RSSI-112; %dbm
     index=index+1;
 end
end
%RSSI=(RSSI_MAX/100)*Signal(j); %Convert to real RSSI 
AP_list{index,count+1}= Signal(j);% RSSI-112; %dbm

%% Finds the quantity of samples per AP
Time_list=cell(1,Num_of_AP);
index=1; % index of each AP
count=0; % AP number
for j=1:len-1
 if strcmp(AP_mac(j),AP_mac(j+1)) == 0
     %RSSI=(RSSI_MAX/100)*Signal(j); %Convert to real RSSI 
     Time_list{index,count+1}= Seconds{j}; % RSSI-112; dbm
     count=count+1;
     index=1;
 else
     %RSSI=(RSSI_MAX/100)*Signal(j); %Convert to real RSSI 
     Time_list{index,count+1}= Seconds{j};% RSSI-112; %dbm
     index=index+1;
 end
end
%RSSI=(RSSI_MAX/100)*Signal(j); %Convert to real RSSI 
Time_list{index,count+1}= Seconds{j};% RSSI-112; %dbm

first=min(cell2mat(Seconds));
last=max(cell2mat(Seconds));
sizeAP=size(AP_list);


Mean=zeros(sizeAP(2),1);
for i=1:sizeAP(2)
	summ=0;
	count=0;
   for j=1:sizeAP(1)
      if ~isempty(AP_list{j,i})
         summ=summ+AP_list{j,i};
         count=count+1;
      end
   end
   Mean(i,1)=summ/count;
end

[A indx]=sort(Mean,'descend');


Final_AP_list = [AP_list(:,indx(1)) AP_list(:,indx(2)) AP_list(:,indx(3)) AP_list(:,indx(4)) AP_list(:,indx(5))];
size_final_list=size(Final_AP_list);
Temp_time=cell(last-first+1,size_final_list(2));
Temp_AP=cell(last-first+1,size_final_list(2));

for i=1:size_final_list(2)
   for k=1:size_final_list(1)
      for j=1:last-first+1
         if isempty(Time_list{k,i})
            continue;
         elseif Time_list{k,i}==j
            Temp_time{j,i}=Time_list{k,i};
            Temp_AP{j,i}=Final_AP_list{k,i};
            break;
         end
      end
   end
end

for i=1:size_final_list(2)
   if isempty(Temp_AP{1,i})
         Temp_AP{1,i}=0;
   end
end

for i=1:size_final_list(2)
   for j=2:last-first+1
      if isempty(Temp_AP{j,i})
         Temp_AP{j,i}=Temp_AP{j-1,i};
      end
   end
end


Final=cell2mat(Temp_AP);

plot(1:last-first+1,Final(:,1),'g');
hold on
plot(1:last-first+1,Final(:,2),'r');
hold on
plot(1:last-first+1,Final(:,3),'b');
% hold on
% plot(1:last-first+1,Final(:,4),'b');
% hold on
% plot(1:last-first+1,Final(:,5)

[ACalcutt]

Glad to hear the beta did what you wanted.

Thanks for posting the matlab code. It would make a good addition to the "Share Code" vistumbler forum if you wanted to add it there.