EBAS 3.00.08 Release Notes

Author:PE/ATMOS <pe@nilu.no>

See also

EBAS Commandline Release Notes

Special Note

Release 3.00.08 is an untypical release. After V.3.00.07, a major V.3.01.00 was planned and development started as a new branch. However 3.01.00 was stuffed with too many new features and development progressed only very slow. Over time, many minor change requests were demanded urgently and development split into a post 3.00.07 agile branch where development cycles were very short and changes pushed into production without tagging and documenting a proper release.

At the end of this phase, before releasing V.3.01.00, this agile changes are retrospectively collected into a formal release with proper documentation.

Features

• All ebas commandline programs: new selection criteria. --diff

  Especially useful for differential extract, but the functionality is available in all commandline programs.

• All ebas commandline programs: new selection criteria --datalevel

  Issue Description: EWB-461

  Fix: svn r1472

• ebas_insert: added option --skip-variables

  Issue Description: EWB-451

  Fix: svn r1441

• ebas_insert: Check for all-missing, all-equal values for a variable are now ERROR instead of WARNING

  Issue Description: EWB-452

  Fix: svn r1442 (bugfixes r1444, r1445)

• ebas_insert: added identical-values to ebas_insert --ignore=identical-values

  Issue Description: EWB-452

  Fix: svn r1442 (bugfixes r1444, r1445)

• ebas_insert: added possibility to automatically ignore identical-values for certain metadata combinations

  Issue Description: EWB-469

  Fix: svn r1535

  currently defined (but may be extended anytime):

  • instrument type optical_particle_size_spectrometer, component

    particle_number_size_distribution and particle size bin > 400 nm

    values may all be 0.00 (no big particles measured)

  • instrument type TEOM and component temperature

    values may be constant

  • data level 0, instrument types dmps and smps

    • component status, temperature, flow_rate or

      particle_diameter

      values may be constant

    • component particle_number_concentration and size bin number > 25

      values may all be 0.0 (no big particles measured)

• ebas_insert: Conversion on import now adds a comment to the metadata

  Issue Description: EWB-454

  Fix: svn r1452

• ebas_insert / ebas_extract: Additional components which are converted on import and export:

  • add conversion for component ethanal

    Issue Description: EWB-453

    Fix: svn r1453

  • add conversion for component temperature matrix met from K to C (import only)

    Issue Description: EWB-465

    Fix: svn r1557

  • add conversion for component SO2 matrix air from nmol/mol to ug S/m3

    Issue Description: EWB-466

    Fix: svn r1551

  • conversion for component ozone: standard temperature changed from 273.15 K to 293.15 K

    Issue Description: EWB-449

    Fix: svn r1450

• ebas_insert: added range check for Volume std. temperature and Volume std. pressure

  Issue Description: EWB-467

  Fix: svn r1517

• ebas_insert: Value/flag consistency: new exceptions for auxiliary datasets

  Issue Description: EWB-411

  Fix: svn r1446

• ebas_insert: all file checks can now make exceptions based on the data level. Needed for level 0 file checks - especially in ebas-submit-tool. We can now, based on data level:

  • accept additional, non standard values for all master data (special parameters only allowed in certain data levels, additional component characteristics, etc.) E.g. component NO_#counts (NOxy level 0 template) or equivalent_black_carbon_loading (filter_absorption_photometer level 0 template)
  • accept additional, non standard metadata elements only allowed in certain data levels. E.g. metadata element Duration of stay in converter or bypass line which is only allowed in the NOxy level 0 template

  Those are exceptions for checking files which are not meant to be stored in the database. The exceptions are only used for the file check routines. Those data can still not be actually stored in the database unless the exceptions are properly implemented in the database.

  Issue Description: EWB-479

  Fix: svn r1563

• ebas_insert: NasaAmes1001: convert no break space automatically to normal blank in all file read operations

  Issue Description: EWB-444

  Fix: svn r1402

• ebas_insert: prevent that the same instrument is used at the same time at different stations

  Issue Description: EWB-458

  Fix: svn r1466

  Backported to V.3.00.07, svn r1469

• ebas_extract: added differential extracts using ebas_extract --diff and ebas_extract --diffxml

  Issue Description: EWB-384

  Fix: svn r1181, r1182, r1187, r1198, r1198, r1202, r1203, r1208

• ebas_extract: added --metadata_option setkey to add ds_setkey to the metadata output.

  Issue Description: EWB-374

  Fix: svn r1164

• ebas_extract: new output file format NetCDF (prototype)

  Issue Description: EWB-314

  Fix: svn r1522

• ebas_list_ds: Data level is now included in the dataset list

  Issue Description: EWB-460

  Fix: svn r1470

• ebas_list_ds: HFI metadata are now included in ebas_list_ds --verbose output (instrument manufacturer, instrument model, instrument serial number)

  Issue Description: EWB-476

  Fix: svn r1543

• ebas_list_ds: Submision metadata are now included in ebas_list_ds --verbose output (revision, submitter, originator, importer)

  Issue Description: EWB-477

  Fix: svn 1559

• new program ebas_plot

Bugfix:

• All ebas commandline programs: time criteria with additional HVM and HDP critera: false positives when HVM and HDP time intervals do not overlap.

  Issue Description: EWB-392

  Fix: svn r1181

• ebas_insert: bug when inserting NRT data before existing data in the data coverage with the same HFI or HVM metadata: HFI and HVM intervals are duplicated (and not aggregated as expected). When exporting those data, multiple export files are generated for the interval fragments.

  Issue Description: EWB-393

  Fix: svn r1172

• ebas_insert: check metadata: length check missing for some string metadata

  Issue Description: EWB-306

  Fix: svn r1317

• ebas_insert: check laboratory code

  Issue Description: EWB-326

  Issue Description: EWB-419

  Fix: svn r1273

• ebas_insert: detection limit and uncertainty in scientific notation does not work

  Issue Description: EWB-425

  Fix: svn r1348, r1349

• ebas_insert: some PS (DO and DS) generated twice

  Issue Description: EWB-432

  Fix: svn r1440

• ebas_insert: finds non-exact matching DS, but does not list them

  Issue Description: EWB-439

  Fix: svn r1386

• ebas_insert: Erroneus boundary check (Missing values are wrongly rejected as below lower bound)

  Issue Description: EWB-442

  Fix: svn r1438

• ebas_insert: crash when reading file without data lines

  Issue Description: EWB-462

  Fix: svn r1476, r1481

• ebas_insert: crash when a newline is found inside a quoted string

  Issue Description: EWB-415

  Fix: svn r1449

• ebas_insert: check sample times not in the future

  Issue Description: EWB-468

  Fix: svn r1518

• ebas_insert: crash when meta data Method ref is missing

  Issue Description: EWB-339

  Fix: svn r1274

  Backported to V.3.00.07, svn r1275

• ebas_insert: file read: flags are rounded in some cases

  Issue Description: EWB-416

  Fix: svn r1264, r1323

  Backported to V.3.00.07, svn r1265, r1324

• ebas_insert: unit conversion crashes

  Issue Description: EWB-397

  Issue Description: EWB-413

  Fix: svn r1241

  Backported to V.3.00.07, svn r1242

• ebas_insert: crash when using 2 different instrument types for the same instrument ref

  Issue Description: EWB-414

  Fix: svn r1246

  Backported to V.3.00.07, svn r1247

• ebas_extract: extracting NRT data: revision date sometimes older than data

  Issue Description: EWB-394

  Fix: svn r1174, r1177

• ebas_extract: crashes on instrument names containing / (slash)

  Issue Description: EWB-320

  Fix: svn r1447

• ebas_extract: flag error when extracting multicolumn

  Issue Description: EWB-440

  Fix: svn r1406

• ebas_extract: crash when converting units on export

  Issue Description: EWB-418

  Fix: svn r1270, r1281

  Backported to V.3.00.07, svn r1271, r1282

• ebas_extract: unit convert: problems with historic data (too little precision)

  Issue Description: EWB-455

  Fix: svn r1451

• ebas_delete: crash when deleting a whole interval including project associations

  Issue Description: EWB-441

  Fix: svn r1392

• ebas_delete: full delete of NRT datasets crashes

  Issue Description: EWB-487

  Fix: svn r1602, r1603

  not completely fixed (workaround for now, final fix is postponed)

• ebas.io: problems when reading ebas nas files with user libraries (ebas-io)

  Issue Description: EWB-396

  Fix: svn r1279

Backported from future releases

• ebas_extract: Format XML: gml:id is sometimes not unique

  Issue Description: EWB-285

  Backported from V.3.01.01, svn r1986, r1988

  Backport: svn r1987, r1989