Additional Data Gathering Tool - v2.6 beta (31 MAR 2009)

[Ian Garton]

What Does this Application do?
Collects flight route information from FlightStats web site for BaseStation flights showing a flight number callsign. Routes collected are stored in to a local SQLite database file (FlightRoutes.sqb) for use with subsequent flights.
Collects photo links for the three most recent images on Airliners web site for each aircraft received by BaseStation. These links are updated monthly for each aircraft.
Collates the number of times each aircraft has been 'seen' on BaseStation, and the date/time and flight route of the last time seen.
Produces daily log reports with flight route data included. These can be viewed, saved or emailed. Automatic daily emailing of log reports is supported.

Important Notes
To view the results of the aircraft photo gathering, or the last seen data collation, you need to replace the default BaseStation.xsl file with the one I've created. Ensure you make a backup of the default BaseStation.xsl file before replacing it should you wish to revert.
The application makes two changes to the structure of the BaseStation database file to allow for storage of flight route details. One change is to add an extra field call 'Route' to the Flights table, and the other is to modify the field length of 'UserTag' in the Aircraft field from 5 to 9 characters. Make a backup copy of your BaseStation.sqb file prior to running this application for the first time in case you wish to revert.
To date, the application hasn't been tested in heavy traffic environments. Here in Brisbane, Australia I get an average of about 15-20 aircraft on my aircraft list, with about 40-50% ADS-B equiped and painting on the radar screen with flight numbers etc. Initially when run, it can take a little while for the application to fill in the details of all current aircraft.

Advanced users may wish to manually edit the FlightRoutes.sqb database and enter flight codes not returned by the FlightStats web site. These are added to the Routes table.
Should you not wish routes to be stored for particular flights, enter the flight callsign in to the DontLogFlights table. This is usefull for flight numbers that have different routes on different days, and will cause a look up of FlightStats web site every time the flight is logged.

Version History/Change Log

v2.6 beta [31/03/09]
* FIXED: Can't Find Table 0 error on startup.
* FIXED: DTD-DTD routes for multi-sector flights.
* CHANGED: Web gathering modified to reflect changes to the FlightStats web site.

v2.5 beta [12/02/09]
* CHANGED: Web gathering modified to reflect changes to the FlightStats web site.

v2.4 beta
* CHANGED: Web gathering modified to reflect changes to the FlightStats web site.

v2.3 beta
* CHANGED: Web gathering modified to reflect changes to the FlightStats web site.

v2.2 beta [28/04/08]
* ADDED: Web proxy function has been added for people blocked by FlightStats.
* CHANGED: A check is in place for routes gathered when the FlightStats web site isn't working properly, or user's IP is blocked.

v2.1 beta [14/04/08]
* ADDED: Option to not use the internet to look up unknown flight numbers has been added.
* CHANGED: Flight routes gathered for multi-segment flights are now suffixed with a ^ to indicate the segment shown may be incorrect.
* CHANGED: Some user interface changes.
* CHANGED: Initial database checking changed to prevent errors.
* CHANGED: Airport Name, and City names are now displayed in proper case rather than full capitals.

v2.0.2 beta [07/03/08]
* FIXED: For some users the 'Close BaseStation' message was appearing on every startup.

v2.0 beta [05/03/08]
* ADDED: Route information can be displayed in either IATA, ICAO, Airport, City or City and Country formats.
* ADDED: Routes can now be gathered if callsign is in IATA as well as ICAO. If the callsign is only numeric, the ICAO code from the Operator Flag Field is used.
* CHANGED: BaseStation gathering now uses socket 30003 stream instead of database polling.
* CHANGED: Routes are now displayed close to immediate.
* CHANGED: ACARSd client rewritten to more efficiently process socket stream.
* CHANGED: Improved route gathering and processing from FlightStats.
* CHANGED: The BaseStation Database 'UserBool' fields are no longer used by this application.
* FIXED: BaseStation.sqb FlightID and AircraftID indexes are re-instated.

v0.8 beta (28/09/07)
* ADDED: ACARSD client included to capture flight callsigns for ModeS aircraft transmitting ACARS.

v0.7 beta (10/09/07)
* CHANGED: Data gathering has been changed to suit new FlightStats site layout.
* CHANGED: Messed up route data entries are removed from the Routes database.

v0.6 beta (10/08/07)
* ADDED: An internal SQLite database file now stores retrieved routes for use when callsign next received.
* ADDED: A program launch command line option of '-start' can be used to auto start gathering when the program is started.
* ADDED: On log reports, a prefixing * now indicates first sighting of an aircraft code.
* CHANGED: Log report formatting slightly modified to improve display when emailed.
* FIXED: Aircraft registrations of 8 characters in length now shown correctly on log reports.
* FIXED: Default value for Interested field is set as No (0) to fix problem when sometimes using Reporter.

v0.5 beta (27/06/07)
* ADDED: Log reports can now be emailed directly from the application.
* ADDED: Log reports can be automatically emailed at a specified time.
* CHANGED: BaseStation database reading/writing has been rewritten to prevent database locking issues, and BaseStation freezing.
* CHANGED: Data gathering has been changed to implemente multi-threading. This prevents the application pausing as in previous versions.
* CHANGED: Minor layout changes to the user interface.
* CHANGED: The image type and internet location of the placeholder Aircraft Details photo has been changed.
* CHANGED: When exiting and then restarting the application, data is now re-gathered for aircraft still on the radar screen.
* CHANGED: On application exit, database fields for aircraft are now cleared of flight route UserTag.
* FIXED: Flight Route gathering has been changed to fix a bug when FlightStats reports the flight as Scheduled.
* FIXED: Log Report layout has been changed to prevent MS Outlook removing line breaks.
* FIXED: Re-instated database indexes and triggers removed by previous versions.
* FIXED: Log Report location field is now saved between application sessions.

v0.4 beta (20/06/07)
* ADDED: BaseStation refresh option to refresh every x seconds.
* ADDED: Saving of screen refresh settings.
* ADDED: Saving of Log Location text.
* ADDED: Filtering implemented on log report to remove flight duplication errors due to known 747-400 callsign issue.
* ADDED: Flight State flag to log report - ARR=Arrival, DEP=Departure, ENR=En Route.
* CHANGED: Flight route gathering algorithm tuned to improve performance.
* CHANGED: Photo gathering is changed to only check for new photos every 30 days, or on every sighting if no photos exist.
* CHANGED: Minor layout changes to user interface.
* CHANGED: Minor layout changes to log report.

v0.3 beta (15/06/07)
Initial Public Release

Files to Download
Please ensure you have read all the above notes before attempting to install these files.
*** Also make sure you have backup copies of your BaseStation.sqb and BaseStation.xsl files. ***

*** Backup your existing FlightRoutes.sqb file if using a previous version of this application. This saves having to look up each flight again. ***

*** For the first time the application is run after installation, please ensure BaseStation and any other SBS-1 addons aren't running. Once the application has started it's ok to then start BaseStation. This is only for the first time it's run after being installed or upgraded from a previous version. ***

Application Installer
http://www.balus.info/BaseStation%20Additional%20Data%20Gathering.msi

Replacement BaseStation.xsl file (Optional)
http://www.balus.info/basestation.xsl

How to make use of this Application
To view the flight route in the BaseStation Aircraft List, right click on the Aircraft List, select Columns, and tick UserTag.



To see the flight route on the BaseStation radar screen, enable viewing of the 'UserTag' field in the 'Scope Fields' tab of the 'Display Settings'.



If using the replacement BaseStation.xsl, when double clicking an aircraft, photos and last seen details are displayed.



Flight routing data is totally dependent on the accuracy of information on FlightStats. Whilst I've programmed in logic to attempt to detect the correct route in cases where the flight is either complex in nature (multi point and/or with different departure date from the present), or the FlightStats data hasn't been updated, incorrect route information will occur from time to time.

[Dave Kenchington]

Hello Ian,
I would be interested in this progam but only for the use of how many times the aircraft has been seen, I have over 2000 sightings p-er day as I live Nr Gatwick, would it be possible just to have this feature alone. As I can see this being useful and save me not purging my data.

Rgds Dave

[RDG]

Congratulations Ian

Your program works a treat and really is a bonus to the basic system.

I run a dual processor system and it does hang for a second or two at times and this may be considerably more on a slower system.

(Added afterwards) Currently have 61 aircraft with 36 on screen so on a busy day that might slow the system up even more.

Will let you know further as time progresses.

Dave

RDG.

[SabreJV]

Hi Ian

I've made an install into a backup version of my system and it is working, I need to restate the warning for anyone thinking of trying this.

As Ian noted you MUST make a backup of your Basestation.sqb file and Basestation.xsl files and keep them safe if you decide to revert back.

Changes are made to the database structure so I'm sure Kinetic will NOT support your system unless you go back to the original database. Let's face it they warned us about changes to Countries.dat and that is not changing the database structure.

----
Sabrejv

[O.W. Wright]

hello

Before i download this i have made a copy of the .sqb and .xsl files.

A couple of questions,

Where should i download this file too, the Basestaion folder or in the User Data folder ?

Do i have to change my .xsl file or can i use this programme to see the flight routing without having to change my .xsl file ?

My .xsl code enables me to display the airline logo , i don't see this in the screenshot of the aircraft details showing three a/c pictures.

If i'm going to change my .xsl then i'd like to keep it similar to what i have now plus the aircraft routing and the three updated aircraft pics.

so what would be the correct sytax for the new code, this is the code i'm using at the moment.

<?xml version="1.0" encoding="iso8859-1"?>
<xsl:stylesheet version="1.0" xmlns:xsl="http://www.w3.org/1999/XSL/Transform">

<xsl:template match="/">
<html>
<head>
<link rel="stylesheet" href="basestation.css" type="text/css"/>
</head>
<body>

<table width="405">
<tr>
<td width="200">
<img src="C:\Program Files\Kinetic\Pictures\{//regdata/registration}.jpg" width="200" border="0" />
</td>
<td width="200" height="100">
<img src="C:\Program Files\Kinetic\Pictures\{//regdata/registration}-1.jpg" width="200" border="0" />
</td>
</tr>
</table>

<table>
<tr>
<td width="200">
<span class="entryheading"></span>
</td>
</tr>
<tr>

<td width="200">
<span class="entrycaption">Flight Number : </span>
<span class="entrydata"><xsl:apply-templates select="//currentflight/callsign"/></span>
</td>
</tr>
<tr>
</tr>
<tr>
</tr>
<tr>
</tr>
<tr>
</tr>
<tr>
</tr>
<tr>
</tr>
<tr>


<td width="400">
<span class="entryheading">Aircraft Data</span>
</td>
</tr>
<tr>
<td>
<div class="entrybox">
<table width="400">
<tr>
<td width="200">
<span class="entrycaption">ModeS Code : </span>
<span class="entrydata"><xsl:apply-templates select="//transmission/modes"/></span>
</td>
<td width="400">
<span class="entrycaption">ICAO Code : </span>
<span class="entrydata"><xsl:apply-templates select="//regdata/ICAOTypeCode"/></span>
</td>


</tr>
<tr>
<td width="200">
<span class="entrycaption">Registration : </span>
<span class="entrydata"><xsl:apply-templates select="//regdata/registration"/></span>
</td>
<td width="400">
<span class="entrycaption">Type : </span>
<span class="entrydata"><xsl:apply-templates select="//regdata/type"/></span>
</td>


</tr>
<tr>
<td width="200">
<span class="entrycaption">Serial No : </span>
<span class="entrydata"><xsl:apply-templates select="//regdata/serialno"/></span>
</td>
<td width="400">
<span class="entrycaption">Operator : </span>
<span class="entrydata"><xsl:apply-templates select="//regdata/registeredowners"/></span>
</td>


</tr>
<tr>
<td width="200">
<span class="entrycaption">Country : </span>
<span class="entrydata"><xsl:apply-templates select="//transmission/country"/></span>
</td>
<td width="400">
<span class="entrycaption">Current Flight No : </span>
<span class="entrydata"><xsl:apply-templates select="//currentflight/callsign"/></span>
</td>


</tr>
<tr>
<td width="200">
<span class="entrycaption">Selcal : </span>
<span class="entrydata"><xsl:apply-templates select="//regdata/status"/></span>
</td>
<td width="400">
<span class="entrycaption">Manufacturer : </span>
<span class="entrydata"><xsl:apply-templates select="//manufacturer"/></span>
</td>


</tr>
<tr>
<td width="200">
<span class="entrycaption">Previous ID : </span>
<span class="entrydata"><xsl:apply-templates select="//previousid"/></span>
</td>
<td width="400">
<span class="entrycaption">Test Reg : </span>
<span class="entrydata"><xsl:apply-templates select="//regdata/genericname"/></span>
</td>


</tr>
<tr>
<td width="200">
<span class="entrycaption">Delivery : </span>
<span class="entrydata"><xsl:apply-templates select="//regdata/currentregdate"/></span>
</td>
<td width="400">
<span class="entrycaption">Year Built : </span>
<span class="entrydata"><xsl:apply-templates select="//yearbuilt"/></span>
</td>


</tr>
<tr>
<td width="200">
<span class="entrycaption">Callsign : </span>
<span class="entrydata"><xsl:apply-templates select="//regdata/popularname"/></span>
</td>



</tr>
<table width="405">
<td width="200" height="50">
<xsl:variable name="AirlineICAO" select="substring(//currentflight/callsign,1,3)"/>
<img src="C:\Program Files\Kinetic\Logos\{$AirlineICAO}.gif" width="150" border="0" />
</td>
</table>

</table>
</div>
</td>
</tr>
</table>

<table>
<tr>
<td width="400">
<span class="entryheading">User Notes</span>
</td>
</tr>
<tr>
<td>
<div class="entrybox">
<table width="400">
<tr>
<td>
<span class="entrydata"><xsl:apply-templates select="//user/notes"/></span>
</td>
</tr>
</table>
</div>
</td>
</tr>
</table>

<table>
<tr>
<td width="400">
<span class="entryheading">Web Links</span>
</td>
</tr>
<tr>
<td>
<div class="entrybox">
<table width="400">
<tr>
<td>

<tr>
<td width="325">
<a href="http://www.airliners.net/search/photo.search?keywords={//regdata/registration}" target="new">
<span class="entrycaption">Airliners.net : <xsl:apply-templates select="//regdata/registration"/></span></a>

</td>
</tr>
<tr>
<td width="300">
<a href="http://www.flytecomm.com/cgi-bin/trackflight?action=select_advanced&amp;flight_id={//currentflight/callsign}" target="new">
<span class="entrycaption">FlyteComm : <xsl:apply-templates select="//currentflight/callsign"/></span></a>
</td>
</tr>
<tr>
<td width="325">
<a href="http://www.jetphotos.net/showphotos.php?offset=0&amp;where=search|-2|-2|-2|-2|{//regdata/registration}|15|1|||||||-2|-2|-2|-2|||15|1||-2|-2||||reg|1||||||-2|||1|||||||" target="new">
<span class="entrycaption">Jetphotos.net : <xsl:apply-templates select="//regdata/registration"/></span></a>
</td>
</tr>
<tr>
<td width="300">
<a href="http://www.planepictures.net/netsearch4.cgi?stype=reg&amp;srng=2&amp;srch={//regdata/registration}&amp;offset=0&amp;range=10" target="new">
<span class="entrycaption">Planepictures.net : <xsl:apply-templates select="//regdata/registration"/></span></a>
</td>
</tr>
<tr>
</tr>
<tr>
</tr>
<tr>
</tr>
<tr>
</tr>
<tr>
</tr>
<tr>

</tr>
</td>
</tr>
</table>
</div>
</td>
</tr>
</table>

</body>
</html>

</xsl:template>
</xsl:stylesheet>

regards

O.W.

[gizmo]

hello Ian...

nice programm the part of collecting the pictures is great stuff...
as Dave i'm also intrested in a "stand alone" aircraft counter (times seen)

i'm running your prog. on a second basestation (just as in the snoopy-test days hi )
thanks for the progie,keep up the good work...

greetzzz Johan...

[holloman]

Running fine here..just a few questions
when it picks up a flight number does it store that flight number for next time i pick that flight number up or does it search again of the internet?..It would be good if it could store the flight routing in a seperate file that could be user editable and be accessed first before it goes looking for it....would save processor speed and hanging!
Also when it stores the links to anet for the photos are they there for good or does next time you start basestation does it look for the photos again?
Thanks
John
PS.
Well done!

[gizmo]

hi Ian...

to avoid the small hick-ups i unticked the automatic refresh field...
i now do an manual refresh every 5 or 10 min.

sbs-populate runs nice aside and looks up the reg's,it's great

greetzzz Johan...

[MikeC]

Thanks Ian, great little app. Well done.

I'm running a dual core 6700 with 2GB ram and a fast graphics card. I have 113 a/c on screen with timeouts as follows:

Position Lost 30
Signal Lost 5
Remove 30
Delete 1800

With the Auto Refresh Basestation checkbox ticked the PC is hanging every 20 seconds for approx 5 seconds.

If I uncheck it, the hanging goes away. What is the refresh doing? Is it the radar screen? If its unchecked, how does one do a manual refresh?

What is the database update counting?

MikeC

[Clunk]

Working well here too, many thanks.

If the photos and flight info were stored for later retrieval without another search on the net, then it would be perfect

[O.W. Wright]

hello


backed up .sqb and .xsl and downloaded the programme but i'm getting an error message.





any help !

[bones]

The application makes two changes to the structure of the BaseStation database file to allow for storage of flight route details. One change is to add an extra field call 'Route' to the Flights table, and the other is to modify the field length of 'UserTag' in the Aircraft field from 5 to 7 characters. Make a backup copy of your BaseStation.sqb file prior to running this application for the first time in case you wish to revert.

This isn't a light step to take. Users should accept that a redesigned SQB file is no longer structured to Kinetic's standard and that they are effectively operating a new database system. This may be incompatible with any future changes Kinetic make to the software.

If users are happy with that and don't mind redesigning the database then accept that Sabre may be right and any support for the database may move from Kinetic to Ian.

[jon01]

i'm getting the same error report as O.W. Wright on running the application

I am using Windows Vista



jon01

[O.W. Wright]

hello jon,

i,m using windows xp

[gizmo]

to Mike C...

you can make an manual refresh by pressing the F5 button,and there is an option in the top bar of basestation: aircraft=>refresh database for all

h.t.h.

greetzzz Johan...