Moment Data Format
==================

The radar produces radar moments which can be consumed by a client application.
The radar produces two streams of data, one for each channel.  The radar
processes a user-specified number of pulses.  The data produced by the radar is
served by the radar using a `nanomsg <http://nanomsg.org>`_ `req
<https://nanomsg.org/v1.1.5/nn_reqrep.html>`_ socket.  The requests can be made
using a :class:`~wni.data_client.DataClient`. of messages serialized with
`msgpack <http://msgpack.org>`_.  The msgpack object is a **list** of **map**
of key-value pairs.  In the case where only a single waveform is being used,
the list will have a length of one.  In the map, each key is a string.    Each
stream of data has the following keys:

.. Note::

    All scan metadata is available via a separate port on the radar: 25004 by
    default.  All scan metadata is valid for the duration of a scan; as long as
    scan_id doesn't change, the metadata is valid.

To work with this data, it may be convenient to use
:class:`~wni.datautil.ScanIter`, and just iterate over ppis or radials as
desired.

+---------------------+------------------------------------------------------------+
| key                 | value                                                      |
+---------------------+------------------------------------------------------------+
| ``scan_id``         | An integer representing the current scan id.  Every time   |
|                     | a new scan is started, this number is incremented by one.  |
+---------------------+------------------------------------------------------------+
| ``radial_number``   | Every output radial is tagged with a monotonically         |
|                     | increasing radial number, which enables thte client to     |
|                     | determine if it is receiving all the data.  If there are   |
|                     | missing numbers, data is being dropped betweeen the radar  |
|                     | and the client program.                                    |
+---------------------+------------------------------------------------------------+
| ``az``              | The azimuth angle, in degrees.                             |
+---------------------+------------------------------------------------------------+
| ``el``              | The elevation angle (this is currently just the actuator   |
|                     | readout)                                                   |
+---------------------+------------------------------------------------------------+
| ``npulses``         | Number of pulses that were used in computing the products. |
+---------------------+------------------------------------------------------------+
| ``channel``         | A number, ``1`` or ``2``, indicating the channel that      |
|                     | produced this set of data.                                 |
+---------------------+------------------------------------------------------------+
| ``timestamp``       | The Unix timestamp, stored as a double.  It is calculated  |
|                     | by a call to ``gettimeofday``.                             |
+---------------------+------------------------------------------------------------+
| ``rev``             | The data revision.  This is a string like ``"1.0.0.beta"`` |
+---------------------+------------------------------------------------------------+
| ``packet_header``   | packet headers, as written by the FPGA. This is useful for |
|                     | debugging purposes.                                        |
+---------------------+------------------------------------------------------------+
| ``ref``             | reflectivity.  This is potentially range corrected if      |
|                     | the scan was set up that way. [#f1]_                       |
+---------------------+------------------------------------------------------------+
| ``velocity``        | Doppler velocity, in m/s. [#f1]_                           |
+---------------------+------------------------------------------------------------+
| ``mag_R1``          | Magnitude of the lag 1 autocorrelation [#f1]_.             |
+---------------------+------------------------------------------------------------+
| ``gates``           | An array of range gates, in meters.  The number of         |
|                     | elements in this array is the same as in the radar         |
|                     | products.                                                  |
+---------------------+------------------------------------------------------------+
| ``iq``              | If ``output_iq`` was selected and you are pulling from     |
|                     | I/Q data server, this will have every pulse of I/Q data    |
|                     | in it.  Otherwise, it will have only the fist pulse of     |
|                     | data.                                                      |
+---------------------+------------------------------------------------------------+
| ``psd``             | If ``output_psd`` and ``doppler_processing`` were enabled  |
|                     | in the scan settings, this contains the entire psd array.  |
|                     | This is useful only for testing and debugging.             |
+---------------------+------------------------------------------------------------+

.. Note::
    All the arrays of data are encoded using msgpack's
    `extension type <https://github.com/msgpack/msgpack/blob/master/spec.md#types-extension-type>`_
    type.  There is a C encoder/decoder for this type and a Python
    encoder/decoder for this type.  In order for a client application to use a
    language other than C or Python, a msgpack decoder for this type must be
    written.  For an implementation in Python for encoding/decoding this
    extended type, see :func:`wni.util.default_pack` and
    :func:`wni.util.ext_hook_unpack`, respectively.

.. rubric:: Footnotes

.. [#f1] ``ref``, ``velocity``, and ``mag_R1`` can be configured to not be
   calculated.  If they were not calculated, the value will be the MessagePack
   `nil` type, which Python interprets as a ``None`` value.