Quick start

Quick start - csv

1. Spin up an EC2 instance on AWS using Geofilter AMI. It takes about 10 minutes for all services to startup.

2. Copy delimited config file to /opt/geofilter_file_inbox.

   scp delimited.conf ubuntu@ec2_server_IP:/opt/geofilter_file_inbox

   Config file contents:

   {
      "files": ["delimited.csv"],  // input data filename. Use "files": "*" to apply a default config to all files
      "format": "delimited",
      "fields": {
         "timestamp": [1, 99],     // [<column, index starts at 0>, <width: ignored>]
         "longitude": [4, 99],
         "latitude": [3, 99],
         "device_id": [2, 99]
      },
      "delimiter": ",",
      "quotechar": '"',
      "timeformat": "iso"
   }
   

2. Copy the sample delimited file to /opt/geofilter_file_inbox.

   scp delimited.csv ubuntu@ec2_server_IP:/opt/geofilter_file_inbox

   The file will be parsed, cleaned and processed in 15 - 30 mins based on Debounce. When status changes to processed, you can pick up the output files from /opt/geofilter_file_outbox.

   Always copy the config file to inbox before transferring the data file. If data file is picked up before config file, you will see failed_parsing messages in the log. Please ignore this.

   You can check the status of file processing by monitoring the contents of the file /opt/geofilter_file_outbox/filemon.log.:

   17-09-2020 15:00:20 delimited.csv:  status => queued
   17-09-2020 15:02:24 delimited.csv:  status => queued
   17-09-2020 15:04:47 delimited.csv:  status => queued
   17-09-2020 15:06:48 delimited.csv:  status => queued
   17-09-2020 15:08:38 delimited.csv:  status => queued
   17-09-2020 15:11:12 delimited.csv:  status => queued
   17-09-2020 15:13:47 delimited.csv:  status => queued
   17-09-2020 15:15:28 delimited.csv:  status => queued
   17-09-2020 15:17:36 delimited.csv:  status => queued
   17-09-2020 15:19:39 delimited.csv:  status => queued
   17-09-2020 15:22:29 delimited.csv:  status => queued
   17-09-2020 15:25:10 delimited.csv:  status => queued
   17-09-2020 15:27:21 delimited.csv:  status => queued
   17-09-2020 15:29:54 delimited.csv:  status => queued
   17-09-2020 15:31:29 delimited.csv:  status => queued
   17-09-2020 15:33:59 delimited.csv:  status => queued
   17-09-2020 15:35:33 delimited.csv:  status => queued
   17-09-2020 15:37:52 delimited.csv:  status => queued
   17-09-2020 15:39:33 delimited.csv: delimited.csv_2020-09-17T14-34-24.060428 status => parsed
   17-09-2020 15:39:33 delimited.csv: delimited.csv_2020-09-17T14-34-24.060428 processed => 3041    total => 3041
   17-09-2020 15:39:33 delimited.csv: delimited.csv_2020-09-17T14-34-24.060428 Writing output files
   17-09-2020 15:39:33 delimited.csv: delimited.csv_2020-09-17T14-34-24.060428 Stream JSON => /opt/geofilter_file_outbox/delimited.csv_2020-09-17T14-34-24.060428_stream.json
   17-09-2020 15:39:34 delimited.csv: delimited.csv_2020-09-17T14-34-24.060428 Stream CSV => /opt/geofilter_file_outbox/delimited.csv_2020-09-17T14-34-24.060428_stream.csv
   17-09-2020 15:39:34 delimited.csv: delimited.csv_2020-09-17T14-34-24.060428 Trips JSON => /opt/geofilter_file_outbox/delimited.csv_2020-09-17T14-34-24.060428_trips.json
   17-09-2020 15:39:34 delimited.csv: delimited.csv_2020-09-17T14-34-24.060428 Processing complete.
   

   File processor produces three output files for each input batch. e.g.;

   • Stream JSON

   • Stream CSV

   • Trips JSON

Quick start - JSON

1. Spin up an EC2 instance on AWS using Geofilter AMI. It takes about 10 minutes for all services to startup.

2. We are going to post JSON data in the format used by React Native Background Geolocation by Transistorsoft - Rnbg is a popular battery efficient location tracking module for IOS & Android. You can configure Rnbg SDK or demo app to post location data continuously to geofilter stream endpoint. When building continuous location tracking apps, use Rnbg’s batch mode to group multiple records into a single HTTP request.

3. Install ReactNative BG Geolocation demo application on an android device.

4. Configure Rnbg demo app to post location data to Geofilter endpoint. Use the public IP address of EC2 server.

   

5. Now go for a drive, walk or run with your android device! Rnbg app will continuously post location data in JSON format to Geofilter EC2 instance. Once you have completed your trip, allow 10 - 15 minutes for records to be processed by the server. Then you can view processed geolocation data using below methods.

6. Grafana dashboard.

   • Url: http://<ec2 public IP>:2325

   • default username: admin

   • default password: Your EC2 instance-id e.g.; i-0830084a61b182663

   1. View and download data as csv.

      

   2. Hourly heatmap - average speed.

      

   3. Frequented locations

      

   4. Location cluster heatmap

      

   5. Cluster hexbins

      

   6. Routes - Ant map

      

7. API

   • Url: http://<ec2 public IP>/<API method>

   1. GET /stream. Processed stream as a geospatial timeseries json. See an example response stream.json

   2. GET /stream/csv. Processed stream as a csv file. See an example response stream.csv

   3. GET /trips?embed=address. Processed stream segmented into trip segments with trip endpoints reverse geocoded into OpenSteetMap street addresses. See an example response trips.json