Evarilos Logo

Evaluation Opportunites ‐ Invitation

EVARILOS Benchmarking Suite

The EVARILOS Benchmarking Suite (EBS) is developed to enable a simple performance evaluation of localization solutions under controlled conditions and following a standardized methodology. The EBS architecture consists of components that implement a workflow that represents three experimentation steps:

  1. In the pre-experimentation step, users can download environment-specific datasets from the appropriate EBS repository, which can be used for training or parameterization of a localization solution. The datasets are available under the ODC Open Database Licence (ODbL).

  2. Evaluation experiments can be orchestrated in the experimentation phase. Tools are available for automatic generation of experiment configurations (e.g. evaluation points, interference patterns). Based on these descriptions, experiment scripts are created and automatically executed.

  3. In the processing phase, the estimated locations can be stored and the performance metrics can be calculated (point accuracy, room-level accuracy, response time, etc). Moreover, experiment results from multiple experiments can be combined for observing how the performance metrics are influenced when individual aspects of different experiments are varied. Finally, an application dependent score can be calculated.

The above process is highly-customizable. All components are available as cloud services, or can be downloaded and modified for custom experimentation purposes. Users are free to use the EBS to realize the full experimentation and performance analysis workflow, or to use only some of the available services. The services are available under the BSD 3-Clause Licence.

Access to the raw datasets, raw data visualization, experimental results and metrics calculation services:

Access to the orchestration of the experiments, experimental results visualization, final scores calculation:

EVARILOS Benchmarking Suite source code download:

FP7 Logo Fire Logo


Report on the results of the real-life experiments in the validation scenarios is available

Report on final results of interference robust localization is available

Final Version of the EVARILOS Benchmarking Handbook is available

Evaluation Opportunities ‐ Technical Annex Track 1 Acrobat PDF Track 2 Acrobat PDF

Open Challenge ‐ Announcement of Winners

Join the EVARILOS mailing list

FIRE web site

FIRE - Brochure (2015)

Next events:
IPSN 2016
11‐14 April, 2016, Vienna
Net Futures 2015
20‐21 April, 2016, Brussels
ICC 2016
23‐27 May, 2016, Kuala Lumpur
IoT Week 2016
31 May‐2 June, 2016, Belgrade
EuCNC 2016
27‐30 June, 2016, Athens
IPIN 2016
4‐7 October, 2016, Madrid

EVARILOS Start: 1.11.2012
Project duration: 30 month
Contract Nr: 317989
EC Contribution: 1.379.944€
Participants: TUB, ADV, iMinds, SICS, THC
Project Fiche Acrobat PDF

This project has received funding from the European Union's Seventh Framework Programme for research, technological development and demonstration under grant agreement no 317989.