.. index::
single: ebas_insert; program
.. program:: ebas_insert
.. _EBASprogram_ebas_insert:
ebas_insert
===========
The program *ebas_insert* imports EBAS NASA Ames files into the EBAS database.
Synopsis
--------
::
ebas_insert.py [-h] [--version] [--cfgfile CFGFILE]
[--loglevconsole LOGLEVCONSOLE]
[--loglevfile LOGLEVFILE] [--logfile LOGFILE]
[--profile] [--nodb] [--dbHost DBHOST] [--db DB]
[--dbUser DBUSER] [--dbPasswd DBPASSWD]
[--transcomment COMMENT] [--nowrite] [--nocommit]
[--ignore IGNORE] [--skip-variables SKIP_VARIABLES]
[--non-interactive] [--nrt] [--email-notify]
[--email-override EMAIL_OVERRIDE]
[--email-bcc EMAIL_BCC] [--email-smtp EMAIL_SMTP]
[filenames [filenames ...]]
Commandline arguments
---------------------
.. include:: ./include/commandline_arguments/intro.include
General arguments
^^^^^^^^^^^^^^^^^
.. include:: ./include/commandline_arguments/general.include
Configuration arguments
^^^^^^^^^^^^^^^^^^^^^^^
.. include:: include/commandline_arguments/config.include
Logging arguments
^^^^^^^^^^^^^^^^^
.. include:: include/commandline_arguments/logging.include
Database arguments
^^^^^^^^^^^^^^^^^^
.. include:: include/commandline_arguments/db.include
.. include:: include/commandline_arguments/db_transcomment.include
.. include:: include/commandline_arguments/db_nowrite.include
.. include:: include/commandline_arguments/db_nocommit.include
Arguments specific to *ebas_insert*
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
.. option:: --ignore=IGNORE
ignore certain errors while reading (numformat, dx0, identical-values)
* ``numformat``
ignore if number format does not match missing value code
* ``dx0``
allow DX=0 with regular data (DX!=0 will always raise an error if not
correct)
* ``identical-values``
ignore errors related to all values
of one variable being equal (e.g. all data missing for
one variable, all values 0.0, or all values any other
constant value). Those cases usually indicate an ERROR
in the data. However if this is investigated and those
data should be read, use this flag to downgrade the
ERROR severity to WARNING. For the opposite (those
cases are definitely identified as ERROR should not be
read while the rest of the file should be read,
consider the --skip-variables parameter)
.. versionadded:: 3.00.08
(default: None)
.. option:: --condense-messages CONDENSE_MESSAGES
.. versionadded:: 3.02.00
Threshold for condensation of multiple occurrences of the same
error or warning message. When specifying 0, the condensation is turned off
and every single message will be propagated. Default: 0
If more then CONDENSE_MESSAGES identical messages (errors or warnings) are
generated, they are condensed. When condensing, the first occurrence of the
message is propagated (printed on console log, written into logfile). A
second message is generated right after the first one, providing additional
information about the repeated (and suppressed) messages.
Example::
ERROR : line 1000: variable[231]: Missing value for flag column is not allowed.
ERROR : line 1000: The previous message repeats 9 times for variable 231. First occurrences in lines: 1000, 1001, 1002, 1003, 1004, ...
.. option:: --skip-variables SKIP_VARIABLES
.. versionadded:: 3.00.08
skip the variable(s) referenced by the variable
number(s) while reading the file.
Specify a single variable number or a comma separated
list of variable numbers. Variable numbers start with
1 for the variable succeeding end_time (same numbering
as in all WARNING/ERROR messages)
start_time and end_time can not be skipped, numflags
may not be skipped
(default: None)
.. option:: --skip-data
.. versionadded:: 3.02.00
Do not read data from the file, only header information and metadata.
This option can be handy for testing purposes: Checking only the header for a
big file ``--skip-data --nowrite``. This way reading is superfast.
If ebas_insert is used to read a file with ``--skip-data``, and no ``nowrite``
is used, The datasets will be created and all metadata information from the
file header is written to the database. The datasets will be created without
any measurement data. In case, the datasets only match partly, the
interactive setup of datasets can be done as usual.
This option is useful when a huge datafile should be scheduled for automatic
import (e.g. during nighttime). In case the datasets do not exist or do not
match exactly, the non-interactive automatic insert would not work. Thus,
the datasets and metadata can be prepared interactive beforehand.
.. option:: --non-interactive
no user interaction if new station or dataset should be created - should be
used in automated processes. (default: False, i.e. interactive behaviour)
.. option:: --nrt
activate NRT operation (DATA WITHOUT HISTORY) (default: False)
.. option:: --email-notify --no-email-notify
.. versionchanged:: 3.02.00 The previos notification functionality (send warnings and errors through email) has been discontinued.
Notify data submitters on successful insert into the database
(default: False)
.. note::
Users who work mainly with operational data management should turn on this
option in their
:ref:`ebas config file <EbasCommandlineConfigurationFile>`, thus ensuring
an email is sent to all data submitters by default.
.. option:: --email-override EMAIL_OVERRIDE
.. versionadded:: 3.02.00
Override the email recipients for notification emails. By default, the email
addresses are extracted from the *data submitters* specified in file's
metadata.
If :option:`--email-override` is specified, no email will be sent to the
*data submitters*, but instead to the email address(es) specified.
It is possible to specify a single email address or a comma separated list of
addresses.
This option can be used instead of totally disabling the notification option.
The email can be forwarded manually later, if needed.
(default: None)
.. option:: --email-bcc EMAIL_BCC
.. versionadded:: 3.02.00
If specified, the notification email will be additionally sent in blind copy
to the email address(es) specified.
This option is very convenient to keep track of all the emails sent to data
submitters.
It is possible to specify a single email address or a comma separated list of
addresses.
(default: <username>@nilu.no)
.. note::
It's highly reccommended for all users to set their correct email address
in their :ref:`ebas config file <EbasCommandlineConfigurationFile>`.
The default <username>@nilu.no does *not* work for all users on all
machines.
.. option:: --email-smtp EMAIL_SMTP
.. versionadded:: 3.02.00
Change the server for sending emails. The default server should usually work,
but in some cases (installations on the internal network or local installation
on a user PC), the email server needs to be changed.
.. option:: filename
input file(s), EBAS NASA-Ames format (default: None)