timezone/timestamp in js/helgoland map client

classic Classic list List threaded Threaded
3 messages Options
Reply | Threaded
Open this post in threaded view
|

timezone/timestamp in js/helgoland map client

danji_ma90
Hi all,

I know this is a very old and very disputed topic in SOS, so it is my pleasure to bring it up once again.

Our SOS instance has the following details:
- SOS bundle version 4.3.8
- Helgoland map client (very recommendable!)
- Records taken in Central European Time (UTC+01:00 Wintertime, UTC+02:00 Summertime)
- Postgres resulttime WITHOUT TIMESTAMP
- Postgres db timezone CET, db server timzone CET
- Webapp server timezone CET

Observation example:
- RAW timestamp (CET): 2014-11-13 15:15
- Postgres database timestamp: 2014-11-13 15:15:00
- Helgoland Client timestamp: 13.11.14 16:15
- Helgoland getData download timestamp: 2014-11-13T16:15:00.000+01:00

As you see the timestamp in both the Helgoland and the downloaded one are incoherent with the original and therefore our SOS data is in a way false data.
I am aware that generally it is recommended to convert the timestamp to UTC before insertion, since SOS assumes the timestamp to be UTC by default, but our amount of data is very big and our dataloggers were always set to CET. I am thinking of a possible solution writing a python script to convert the timestamps of our data to UTC, but thought I'd ask around if anyone has had the same issue and found a better solution to the problem. Is there any way of SOS to use the raw timestamp?
If not, does anyone know of an effective way to convert the timestamps to UTC correctly in our CSV data (considering that UTC doesn't use Summer/Winter time shifts)?

Thanks for the help,

Daniel

Reply | Threaded
Open this post in threaded view
|

Re: timezone/timestamp in js/helgoland map client

Henning Bredel-2
If you are able to upgrade your SOS instance to 4.3.12 you'll find a timezone config for the data source (database). 

So when having an offset in your time data while storing it as "timestamp without timezones" the SOS will take care.

Best

  Henning

Am 09.05.2017 17:52 schrieb "danji_ma90" <[hidden email]>:
Hi all,

I know this is a very old and very disputed topic in SOS, so it is my
pleasure to bring it up once again.

Our SOS instance has the following details:
- SOS bundle version 4.3.8
- Helgoland map client (very recommendable!)
- Records taken in Central European Time (UTC+01:00 Wintertime, UTC+02:00
Summertime)
- Postgres resulttime WITHOUT TIMESTAMP
- Postgres db timezone CET, db server timzone CET
- Webapp server timezone CET

Observation example:
- RAW timestamp (CET): 2014-11-13 15:15
- Postgres database timestamp: 2014-11-13 15:15:00
- Helgoland Client timestamp: 13.11.14 16:15
- Helgoland getData download timestamp: 2014-11-13T16:15:00.000+01:00

As you see the timestamp in both the Helgoland and the downloaded one are
incoherent with the original and therefore our SOS data is in a way false
data.
I am aware that generally it is recommended to convert the timestamp to UTC
before insertion, since SOS assumes the timestamp to be UTC by default, but
our amount of data is very big and our dataloggers were always set to CET. I
am thinking of a possible solution writing a python script to convert the
timestamps of our data to UTC, but thought I'd ask around if anyone has had
the same issue and found a better solution to the problem. Is there any way
of SOS to use the raw timestamp?
If not, does anyone know of an effective way to convert the timestamps to
UTC correctly in our CSV data (considering that UTC doesn't use
Summer/Winter time shifts)?

Thanks for the help,

Daniel





--
View this message in context: http://sensorweb.forum.52north.org/timezone-timestamp-in-js-helgoland-map-client-tp4028941.html
Sent from the 52° North - Sensor Web Community Forum mailing list archive at Nabble.com.
_______________________________________________
SWE mailing list
[hidden email]
http://list.52north.org/mailman/listinfo/swe
http://sensorweb.forum.52north.org
Please respect our mailing list guidelines:
http://52north.org/resources/mailing-lists-and-forums/guidelines

_______________________________________________
SWE mailing list
[hidden email]
http://list.52north.org/mailman/listinfo/swe
http://sensorweb.forum.52north.org
Please respect our mailing list guidelines:
http://52north.org/resources/mailing-lists-and-forums/guidelines
Reply | Threaded
Open this post in threaded view
|

Re: timezone/timestamp in js/helgoland map client

danji_ma90
Thank you very much, the implementation of the timezone configuration was really important to avoid timestamp errors. I noticed though you have to use it with caution:

Since this new function is implemented in the API and SOS still assumes the dataset timestamps are in UTC from the start, it will add the time difference between UTC and the datasource timezone defined when installing the web application, meaning the timestamps in the db will be modified.
eg: Our timestamps are without defined timezone and recorded in CET. When setting the datasource timezone to CET, InsertObservation will add one/two hours (depending on summer or wintertime) to the timestamp and store the result in the database. Because our server is located in CET timezone, the SOS map client will not modify the timestamp in the online map client (helgoland), but the timestamp wil be false due to the timestamp modification when inserting the data.

To avioid this problem the datasource timezone should be set to UTC when doing InsertObservation posts (so the timestamps are stored exactly as in our CSV files), but to the local timezone (in our case CET) when doing Get requests. We found a way to do this, installing one SOS webapp for inserting data with UTC and another for get requests/map visualisation/downloading the data with datasource timezone CET.

Nevertheless, the new function works very well and the 52North work is strongly appreciated. Also including helgoland to the standard SOS4.3.12 bundle drammatically improves the webapp's user friendliness.