Issue with UNIX Time - SOS Importer

classic Classic list List threaded Threaded
6 messages Options
Reply | Threaded
Open this post in threaded view
|

Issue with UNIX Time - SOS Importer

kanishk3687
Hello,

I am using SOS Importer (https://wiki.52north.org/bin/view/SensorWeb/SosImporter) to import observations from a CSV file. The Date & Time format in my CSV file is in UNIX Time.  However, it fails to import and gives the error message, stating unparseable date. I selected appropriate option 'UNIX Time' in step 3 of SOS Import process.

Could you please suggest, how can I import UNIX Time using SOS Importer. I have pasted complete error message below.

Thanks & best regards,
Kanishk

--------------------------------------
15:32:42.138 [OneTimeFeeder] DEBUG SensorObservationService - HeapSize Information: max: 247MB; total now: 15MB; free now: 2MB; used now: 13MB
15:32:42.138 [OneTimeFeeder] DEBUG SensorObservationService - Handling CSV line #2: [Wind Speed, 1465468240, 1.610000, Kph]
15:32:42.138 [OneTimeFeeder] DEBUG SensorObservationService - Parsing measured value column 2
15:32:42.139 [OneTimeFeeder] DEBUG SensorObservationService - Sensor: Sensor [name=Offering_mySensor, uri=Offering_mySensor]
15:32:42.141 [OneTimeFeeder] DEBUG SensorObservationService - Feature of Interest: FeatureOfInterest [p=Position [values=[-0.011892000213265419, 51.53843688964844, 0.0], units=[deg, deg, meters], epsgCode=4326], name=mySensor_London, uri=mySensor_London]
15:32:42.141 [OneTimeFeeder] DEBUG SensorObservationService - Value: 1.61
15:32:42.142 [OneTimeFeeder] ERROR SensorObservationService - Could not retrieve all information required for insert observation because of parsing error: java.text.ParseException: Unparseable date: "1465468240". Skipped this one.
15:32:42.143 [OneTimeFeeder] DEBUG SensorObservationService - Exception stack trace:
java.text.ParseException: Unparseable date: "1465468240"
        at java.text.DateFormat.parse(Unknown Source) ~[na:1.8.0_91]
        at org.n52.sos.importer.feeder.DataFile.parseTimestampComponent(DataFile.java:846) ~[52n-sos-importer-feeder-0.5.0-20150807.134409-5-bin.jar:0.5.0-SNAPSHOT]
        at org.n52.sos.importer.feeder.DataFile.getTimeStamp(DataFile.java:418) ~[52n-sos-importer-feeder-0.5.0-20150807.134409-5-bin.jar:0.5.0-SNAPSHOT]
        at org.n52.sos.importer.feeder.SensorObservationService.getInsertObservationForColumnIdFromValues(SensorObservationService.java:585) [52n-sos-importer-feeder-0.5.0-20150807.134409-5-bin.jar:0.5.0-SNAPSHOT]
        at org.n52.sos.importer.feeder.SensorObservationService.getInsertObservations(SensorObservationService.java:557) [52n-sos-importer-feeder-0.5.0-20150807.134409-5-bin.jar:0.5.0-SNAPSHOT]
        at org.n52.sos.importer.feeder.SensorObservationService.importData(SensorObservationService.java:302) [52n-sos-importer-feeder-0.5.0-20150807.134409-5-bin.jar:0.5.0-SNAPSHOT]
        at org.n52.sos.importer.feeder.task.OneTimeFeeder.run(OneTimeFeeder.java:203) [52n-sos-importer-feeder-0.5.0-20150807.134409-5-bin.jar:0.5.0-SNAPSHOT]
        at java.lang.Thread.run(Unknown Source) [na:1.8.0_91]
Reply | Threaded
Open this post in threaded view
|

Re: Issue with UNIX Time - SOS Importer

Eike Hinderk Jürrens
Dear kanishk,
could you please provide your configuration and lines of the csv files
for testing using any pastebin service.
Kind regards,
Eike

On 22.06.2016 16:05, kanishk3687 wrote:

> Hello,
>
> I am using SOS Importer
> (https://wiki.52north.org/bin/view/SensorWeb/SosImporter) to import
> observations from a CSV file. The Date & Time format in my CSV file is in
> UNIX Time.  However, it fails to import and gives the error message, stating
> *unparseable date*. I selected appropriate option 'UNIX Time' in step 3 of
> SOS Import process.
>
> Could you please suggest, how can I import UNIX Time using SOS Importer. I
> have pasted complete error message below.
>
> Thanks & best regards,
> Kanishk
>
> --------------------------------------
> /15:32:42.138 [OneTimeFeeder] DEBUG SensorObservationService - HeapSize
> Information: max: 247MB; total now: 15MB; free now: 2MB; used now: 13MB
> 15:32:42.138 [OneTimeFeeder] DEBUG SensorObservationService - Handling CSV
> line #2: [Wind Speed, 1465468240, 1.610000, Kph]
> 15:32:42.138 [OneTimeFeeder] DEBUG SensorObservationService - Parsing
> measured value column 2
> 15:32:42.139 [OneTimeFeeder] DEBUG SensorObservationService - Sensor: Sensor
> [name=Offering_mySensor, uri=Offering_mySensor]
> 15:32:42.141 [OneTimeFeeder] DEBUG SensorObservationService - Feature of
> Interest: FeatureOfInterest [p=Position [values=[-0.011892000213265419,
> 51.53843688964844, 0.0], units=[deg, deg, meters], epsgCode=4326],
> name=mySensor_London, uri=mySensor_London]
> 15:32:42.141 [OneTimeFeeder] DEBUG SensorObservationService - Value: 1.61
> 15:32:42.142 [OneTimeFeeder] ERROR SensorObservationService - Could not
> retrieve all information required for insert observation because of parsing
> error: java.text.ParseException: Unparseable date: "1465468240". Skipped
> this one.
> 15:32:42.143 [OneTimeFeeder] DEBUG SensorObservationService - Exception
> stack trace:
> java.text.ParseException: Unparseable date: "1465468240"
> at java.text.DateFormat.parse(Unknown Source) ~[na:1.8.0_91]
> at
> org.n52.sos.importer.feeder.DataFile.parseTimestampComponent(DataFile.java:846)
> ~[52n-sos-importer-feeder-0.5.0-20150807.134409-5-bin.jar:0.5.0-SNAPSHOT]
> at org.n52.sos.importer.feeder.DataFile.getTimeStamp(DataFile.java:418)
> ~[52n-sos-importer-feeder-0.5.0-20150807.134409-5-bin.jar:0.5.0-SNAPSHOT]
> at
> org.n52.sos.importer.feeder.SensorObservationService.getInsertObservationForColumnIdFromValues(SensorObservationService.java:585)
> [52n-sos-importer-feeder-0.5.0-20150807.134409-5-bin.jar:0.5.0-SNAPSHOT]
> at
> org.n52.sos.importer.feeder.SensorObservationService.getInsertObservations(SensorObservationService.java:557)
> [52n-sos-importer-feeder-0.5.0-20150807.134409-5-bin.jar:0.5.0-SNAPSHOT]
> at
> org.n52.sos.importer.feeder.SensorObservationService.importData(SensorObservationService.java:302)
> [52n-sos-importer-feeder-0.5.0-20150807.134409-5-bin.jar:0.5.0-SNAPSHOT]
> at
> org.n52.sos.importer.feeder.task.OneTimeFeeder.run(OneTimeFeeder.java:203)
> [52n-sos-importer-feeder-0.5.0-20150807.134409-5-bin.jar:0.5.0-SNAPSHOT]
> at java.lang.Thread.run(Unknown Source) [na:1.8.0_91]/
>
>
>
> --
> View this message in context: http://sensorweb.forum.52north.org/Issue-with-UNIX-Time-SOS-Importer-tp4028542.html
> Sent from the 52° North - Sensor Web Community Forum mailing list archive at Nabble.com.
> _______________________________________________
> SWE mailing list
> [hidden email]
> http://list.52north.org/mailman/listinfo/swe
> http://sensorweb.forum.52north.org
> Please respect our mailing list guidelines:
> http://52north.org/resources/mailing-lists-and-forums/guidelines


--
Eike Hinderk Jürrens
52°North Initiative for Geospatial Open Source Software GmbH
Martin-Luther-King-Weg 24
48155 Münster, Germany
E-Mail: [hidden email]
Fon: +49-(0)-251–396371-33
Fax: +49-(0)-251–396371-11
http://52north.org/
Twitter: @FiveTwoN
General Managers: Dr. Albert Remke, Dr. Andreas Wytzisk
Local Court Muenster HRB 10849

_______________________________________________
SWE mailing list
[hidden email]
http://list.52north.org/mailman/listinfo/swe
http://sensorweb.forum.52north.org
Please respect our mailing list guidelines:
http://52north.org/resources/mailing-lists-and-forums/guidelines
Reply | Threaded
Open this post in threaded view
|

Re: Issue with UNIX Time - SOS Importer

kanishk3687
Hi Eike,

thank you for your response. The CSV lines are as follows:
Wind Speed,1465468180.561000,4.830000,Kph
Wind Speed,1465468240.993000,1.610000,Kph


which are in the order
ObservationPropertyName, UnixTime, Value, Unit

My query is, SOS Importer is unable to parse Unix Time (even if I remove milliseconds). When I change this timestamp to normal ISO one (e.g. YYYY-MM-DD HH:MM:SS), it works perfectly fine.

Could you just let me know, if it is possible to import Unix Timestamps using SOS Importer.

Best regards,
Kanishk
Reply | Threaded
Open this post in threaded view
|

Re: Issue with UNIX Time - SOS Importer

Matthias Müller
Kanisk,

Looks like your Unix time stamps are in fact decimals, not integers. This might cause the problem. Have you tried to truncate decimal places, i.e.:

1465468240.993000 -> 1465468240

Matthias


-----Original Message-----
From: SWE [mailto:[hidden email]] On Behalf Of kanishk3687
Sent: Friday, June 24, 2016 10:21 AM
To: [hidden email]
Subject: Re: [52N SWE] Issue with UNIX Time - SOS Importer

Hi Eike,

thank you for your response. The CSV lines are as follows:
/Wind Speed,1465468180.561000,4.830000,Kph
Wind Speed,1465468240.993000,1.610000,Kph/

which are in the order
/ObservationPropertyName, UnixTime, Value, Unit/

My query is, SOS Importer is unable to parse Unix Time (even if I remove milliseconds). When I change this timestamp to normal ISO one (e.g.
YYYY-MM-DD HH:MM:SS), it works perfectly fine.

Could you just let me know, if it is possible to import Unix Timestamps using SOS Importer.

Best regards,
Kanishk



--
View this message in context: http://sensorweb.forum.52north.org/Issue-with-UNIX-Time-SOS-Importer-tp4028542p4028548.html
Sent from the 52° North - Sensor Web Community Forum mailing list archive at Nabble.com.
_______________________________________________
SWE mailing list
[hidden email]
http://list.52north.org/mailman/listinfo/swe
http://sensorweb.forum.52north.org
Please respect our mailing list guidelines:
http://52north.org/resources/mailing-lists-and-forums/guidelines

_______________________________________________
SWE mailing list
[hidden email]
http://list.52north.org/mailman/listinfo/swe
http://sensorweb.forum.52north.org
Please respect our mailing list guidelines:
http://52north.org/resources/mailing-lists-and-forums/guidelines

smime.p7s (8K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Issue with UNIX Time - SOS Importer

kanishk3687
Hi Matthias,

yes - I have also tried to truncate decimal places. It shows the same error message. When we select 'Unix Time' option in SOS Importer (Step 3a), it doesn't parse the timestamps. On the other hand, if I use ISO timestamps and select 'Combination' option (Step 3a), it gives a message in blue stating, 'All values could be interpreted'.

I am not sure, if it is able to read Unix Timestamps (I could not find any example for the same in SOS Importer as well).

Best,
Kanishk
Reply | Threaded
Open this post in threaded view
|

Re: Issue with UNIX Time - SOS Importer

Eike Hinderk Jürrens
Hi Kanishk and Matthias,
sorry for the long delay. The issue is closed:

    https://github.com/52North/sos-importer/issues/66

Kind regards,
Eike

On 24.06.2016 11:59, kanishk3687 wrote:

> Hi Matthias,
>
> yes - I have also tried to truncate decimal places. It shows the same error
> message. When we select 'Unix Time' option in SOS Importer (Step 3a), it
> doesn't parse the timestamps. On the other hand, if I use ISO timestamps and
> select 'Combination' option (Step 3a), it gives a message in blue stating,
> 'All values could be interpreted'.
>
> I am not sure, if it is able to read Unix Timestamps (I could not find any
> example for the same in SOS Importer as well).
>
> Best,
> Kanishk
>
>
>
> --
> View this message in context: http://sensorweb.forum.52north.org/Issue-with-UNIX-Time-SOS-Importer-tp4028542p4028550.html
> Sent from the 52° North - Sensor Web Community Forum mailing list archive at Nabble.com.
> _______________________________________________
> SWE mailing list
> [hidden email]
> http://list.52north.org/mailman/listinfo/swe
> http://sensorweb.forum.52north.org
> Please respect our mailing list guidelines:
> http://52north.org/resources/mailing-lists-and-forums/guidelines


--
Eike Hinderk Jürrens
52°North Initiative for Geospatial Open Source Software GmbH
Martin-Luther-King-Weg 24
48155 Münster, Germany
E-Mail: [hidden email]
Fon: +49-(0)-251–396371-33
Fax: +49-(0)-251–396371-11
http://52north.org/
Twitter: @FiveTwoN
General Managers: Dr. Albert Remke, Dr. Andreas Wytzisk
Local Court Muenster HRB 10849

_______________________________________________
SWE mailing list
[hidden email]
http://list.52north.org/mailman/listinfo/swe
http://sensorweb.forum.52north.org
Please respect our mailing list guidelines:
http://52north.org/resources/mailing-lists-and-forums/guidelines