DATA FLOWS

(IN) Inbound DATA flows

A possibility of plugging to external telemetry data sources and having them fed into Anitra

Note the devices imported from the data source are imported to your profile and listed in the device list. A unified metadata structure can be then created on top of the imported data (Device - Tracking - Animal). Ultimately data from various sources can be browsed and visualized side by side in a unified metadata-rich structure

Connectors for the following data sources currently exist:

• Movebank
• Ecotone
• Ornitela

Inbound connections are managed in the "Integrations In" section. The overview of existing connections allows adding, editing/removal and running/scheduling the connection

Use the "+create new" button to launch the connector creation wizard and enter the information required for running the feed

Note each of the connectors works differently and hence also the information needed for running the data flow and scheduling the run slightly differs.

Ecotone

• details needed: Ecotone panel name (=URL of CSV files storage), user name, and password
• user can manage this connector fully i.e. create, edit, run (once or schedule regular run), and delete
  

Ornitela

• details needed: Ornitela user name and password
• for Ornitela the data integration relies on Ornitela SFTP functionality.
• i.e. in this case user creates, edits the data connector, and requests activation (or requests deactivation) of the data flow
• by requesting activation user also requests to have the SFTP configuration in his Ornitela pannel configured by us.
  
• Note some related aspects using SFTP:
  • only new data are integrated once the data flow is activated.
  • for importing older data (records created before the feed activation) use import functionality (any duplicate records are merged or skipped so you can import the same period repeatedly).
  • only the devices marked for SFTP export in the Ornitela panel will be automatically updated into Anitra
  • make sure you also update the connector definition anytime you are changing your Ornitela user password
  • also bear in mind that modifying the list of devices marked for SFTP (unchecking and checking again) will result in the data flow interruption and thus to some possible gaps in the data.

Movebank to Anitra

• configuring this connector a Movebank data + Movebank metadata are connected
• Movebank API is employed for accessing the records
• in this case user is supposed to enter a movebank username and password in combination with Movebank Study ID and request the data flow activation
  

(IN) Manual data import

Data Import dialog for orgaanisation level data can be accessed in Data management section of your customer profile

Import dialog now accepts data in the following structure/teplates:

1) data mapped directly to Anitra database column names:

• labeled as a "Basic format" file type in the Import dialog
• download this template file listing available columns and sample values to be used

2) data in the original file structure and format of Anitra and some other tag producers (fixed custom mapping of original files to Anitra database).

• labeled as "Anitra CSV format"
• labeled as "Ornitela CSV format" (version 1 file)
• labeled as "Ecotone CSV format"

It allows importing telemetry data in the original file structure and format generated by tag producers control pannels These flows can be useful when uploading older data files not available anymore to automated flow e.g. older records after having activated Ornitela SFTP data flow (which only generated new records).

Note this general behavior traits of the import flow

• import procedure performs append and merge data operations i.e.
  • for any new device (device_code) detected in the file a new Device will be created in Anitra and all the imported rows linked to it
  • for devices matching existing Anitra devices the data from the file will be merged (rows inserted or updated using data timestamp, no delete occures here)

• first step of the import flow evaluates the file structure and the data contained
  • completely wrong files will display erros and will not be imported. following type of problems will prevent file from being imported
    • wrong column separators,
    • unmatching names of key columns (such as device identification, dateime, or latitude and longitude or containi)
    • duplicate column names in the list
    • wrong coding used
      
  • some less severe problems will only result in having some columns from the file skipped and not imported. e.g.missing column name or wrong name (not exactly matching teh template columns)
    

• similarly as for the file structure the rows are also checked. The rows with problem data are detected, problems shown and rows skipped while importing. You can possibly notice these
  • errors:
    • data in the cell not matching the expected target format (e.g. wrong date time format, string data too long, text data in numeric columns or similar datatype conflicts etc)
    • all these rows will be skipped during the import
      
  • warnings:
    • possible data loss by overwritiong existing data (same combination of device and date time already exists in the database)
    • these row will be used to update existing data base records
      

Note

• any user (inculding those not involved in any Customer account) can upload data under their private user profile (ask us in case you need this feature activated for your user account)
  • in this case devices created are stored in users profile only and have no relationship with any Customer/Organisation entity
  • the user who uploaded the data becomes "Device Owner"
• for importing data in the particular customer profile (not in personal profile) make sure you chose one of your organization accounts in the "Upload in profile" selection
  • in this case data imported will be incorporated under the selected "Customer/Organisation" entity

(OUT) Anitra API data & metadata access

• allows grabbing Anitra DATA and METADATA from external tools such as from GIS systems API interface
• curent version of API offers access to the following items:
  • Device List (=Device metadata)
  • Device Data
  • Tracking List (=Tracking and Animal metadata)
  • Tracking Data
• for detailed documentation and examples of API implementation refer to Anitra Public API documentation
• note the API is currently linked to user access rights. Therefore your API credentials can be managed down in your user profile "API keys" section. You can generate new keys or revoke existing here.
  

(OUT) Anitra to Movebank data feed

• Anitra to Movevebank data feed can be set up in the "integration out" section of the customer profile using the "+create new" button
  
• a Movebank data owner username needs to be specified.
• the account provided will be used to host all the synchronized data in Movebank
  
• you can switch ON/OFF particular tag data synchronization by the "Export to Movebank" checkbox in the device profile.
• by default the synchronization is inactive for all the new devices
  
• you can review the status of the check and the last time of synchronization of each of your devices at the device list

(OUT) Manual DATA Export

• allows manually exporting data from Anitra to CSV, XLSX, KML, and GPX formats for offline use

• export feature can be launched in various contexts / places of the platform:

1. Customer level (data integration section of customer profile) - full data for all ANITRA devices owned by Customer are bulk exported in this case.
   
   Note having collected lot of data, this type of export is prone to fail - for mass downloading all your data we strongly recomend using our API functionality instead!.
2. Device level - the "Export" button in Device profile or using context menu in Device grid.
   
3. Tracking level - the "Export" button in Animal/Tracking profile or using context menu in Tracking grid
   

• exported data can be filtered by a date range
• the column editor feature allows modifying output columns in terms of columns list, column order, and naming of exported columns (a user preference)
  

• Note the data export functionality is typicaly only available for device admins users (Customer/Device Owner/Admins). Users with shared access ("view only") are not supposed to download and store the shared records outside of the system. In case of need a "share with data export" can be used.
  

(OUT) Manual METADATA Export

• allows manually exporting Device, Tracking and Animal metadata from Anitra to CSV and XLSX formats
• this export feature can be launched in Device and Tracking list using items from the context dialogue
  
• you can modify the columns that will be exported (column list, order, names) using "column editor"
• use the item "edit columns" from context dialogue over the Device or Tracking grid header line and switch to "export grid" configuration