QGIS SOS Plugin

classic Classic list List threaded Threaded
9 messages Options
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

QGIS SOS Plugin

Lionel Menard
Hi,
we have some interests in accessing SOS resources using QGIS. I've tested the following plugin:
https://github.com/ruben-mv/SOSClient

I've get some pb. trying to use it and though I can retrieve the GetCapabilities information from a remote SOS, I cannot draw station location on a map and plot the observation.

I reported such pb. on the GitHub page of the plugin developers:
https://github.com/ruben-mv/SOSClient/issues/9

Does any of you have already tested and successfully draw and plot SOS resources using this plugin ?

Do you know of a working QGIS based plugin to access SOS resources ?

Thanks for your help.

Lionel
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: QGIS SOS Plugin

Lionel Menard
This post is definitely not moving forward (Even in term of view...) :-(

Let me elaborate a bit more about it.

It is important at some point toward our community (renewable energy) to have a message about interoperability regarding in-situ measurements and more precisely SOS. We all know that maps and layers interoperable processes work well now (e.g.. WMS, WFS, even WCS). They are on the table sinces many years.
In order to convince the community to put efforts (time and money) to release their data respecting such standard approach we need to illustrate the benefit of doing so in the largest possible ways.

Displaying the data into a client (e.g.. The JavaScript API) is a absolutely needed step. But despite the fact the default API based access this not a per say SOS standard service access (I know it is possible of doing so in the Client), this is definitely not enough for sustaining a speech about the ubiquity of the service oriented architecture approach.

I know there are other way to illustrate this and I've seen (not tested) the ArcGIS plugin that tho in that direction (use of standard) even though this is not based on a free and Open Source platform.

A lot of our targeted practitioners for Renewable Energy applications use Open Source tools such a QGIS for integrating resources such as maps, layers, and of course in-situ measurements (currently using local flat file) for their own business or research. Therefore having a working plugin for one of the main Open source GIS platform could definitely leverage the interest of providing such effort to enable access to service based in-situ measurement. 52N solution being one the highly ranked choice for supporting the SOS side.

I've already discussed with some in-situ data integrator regarding SOS in-situ access into QGIS and they were on the same line as I do.

I do not precisely know the usage of such mix between map/layers based information and in-situ measurements around the community that use 52N solution, but I would be delighted to ear from you about this.

Do you have such need ?
Has any of you tested the mentioned plugin ?
Are you successfully using the ArcGIS one ?
Do you have any knowledge or idea to make this plugin works for 52N SOS resources ?

Thanks for reading and for your further comments.

Cheers,

Lionel
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: QGIS SOS Plugin

Just van den Broecke-2
On 17-12-15 14:24, Lionel Menard wrote:

> This post is definitely not moving forward (Even in term of view...) :-(
>
> Let me elaborate a bit more about it.
>
> It is important at some point toward our community (renewable energy) to
> have a message about interoperability regarding in-situ measurements and
> more precisely SOS. We all know that maps and layers interoperable processes
> work well now (e.g.. WMS, WFS, even WCS). They are on the table sinces many
> years.
> In order to convince the community to put efforts (time and money) to
> release their data respecting such standard approach we need to illustrate
> the benefit of doing so in the largest possible ways.
>
> Displaying the data into a client (e.g.. The JavaScript API) is a absolutely
> needed step. But despite the fact the default API based access this not a
> per say SOS standard service access (I know it is possible of doing so in
> the Client), this is definitely not enough for sustaining a speech about the
> ubiquity of the service oriented architecture approach.
>
> I know there are other way to illustrate this and I've seen (not tested) the
> ArcGIS plugin that tho in that direction (use of standard) even though this
> is not based on a free and Open Source platform.
>
> A lot of our targeted practitioners for Renewable Energy applications use
> Open Source tools such a QGIS for integrating resources such as maps,
> layers, and of course in-situ measurements (currently using local flat file)
> for their own business or research. Therefore having a working plugin for
> one of the main Open source GIS platform could definitely leverage the
> interest of providing such effort to enable access to service based in-situ
> measurement. 52N solution being one the highly ranked choice for supporting
> the SOS side.
>
> I've already discussed with some in-situ data integrator regarding SOS
> in-situ access into QGIS and they were on the same line as I do.
>
> I do not precisely know the usage of such mix between map/layers based
> information and in-situ measurements around the community that use 52N
> solution, but I would be delighted to ear from you about this.
>
> Do you have such need ?
> Has any of you tested the mentioned plugin ?
> Are you successfully using the ArcGIS one ?
> Do you have any knowledge or idea to make this plugin works for 52N SOS
> resources ?
>
> Thanks for reading and for your further comments.
>
> Cheers,
>
> Lionel
>
>
>
> --
> View this message in context: http://sensorweb.forum.52north.org/QGIS-SOS-Plugin-tp4028298p4028314.html
> Sent from the 52° North - Sensor Web Community Forum mailing list archive at Nabble.com.
> _______________________________________________
> SWE mailing list
> [hidden email]
> http://list.52north.org/mailman/listinfo/swe
> http://sensorweb.forum.52north.org
> Please respect our mailing list guidelines:
> http://52north.org/resources/mailing-lists-and-forums/guidelines
>

_______________________________________________
SWE mailing list
[hidden email]
http://list.52north.org/mailman/listinfo/swe
http://sensorweb.forum.52north.org
Please respect our mailing list guidelines:
http://52north.org/resources/mailing-lists-and-forums/guidelines
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: QGIS SOS Plugin

Just van den Broecke-2
In reply to this post by Lionel Menard
(sorry for the empty first post)

Yes, we (Geonovum, The Netherlands) have interest in this plugin!

I have been playing with it, opened some issues and made some Python
tweaks to have it at least communicating with the 52N SOS. Will issue
PRs. Still quite some effort is required in making the current
functionality work and the more for additional functions. We are looking
into getting resources/funds to have it developed further. I think for
the original developer it was a one-shot graduation project. Like with
all Open Source it is up to us, the community, to bring this plugin
further based on our needs.

One of the main requirements from our users is to be able to locally
download/export timeseries for further analysis.

A problem with SOS (like WFS) is that there may be potentially huge
amounts of data (observations), in our case millions of observations,
linked to a few FOIs. Like it is now the plugin/QGIS will lock-up. That
needs to be catered for, possibly with paging. We prefer the (52N) SOS
REST API even, but that may not be standard.

So I acknowledge the need for a QGIS SOS Plugin, the more now SOS may be
part of INSPIRE Download Services. The constraints will be interesting
as lot of QGIS is layer/feature based. How to fit SOS into this concept
may be a challenge. On the other hand the Python plugins can be versatile.

Ok, this mail is getting too long already :-). best,

Just van den Broecke
The Netherlands


On 17-12-15 14:24, Lionel Menard wrote:

> This post is definitely not moving forward (Even in term of view...) :-(
>
> Let me elaborate a bit more about it.
>
> It is important at some point toward our community (renewable energy) to
> have a message about interoperability regarding in-situ measurements and
> more precisely SOS. We all know that maps and layers interoperable processes
> work well now (e.g.. WMS, WFS, even WCS). They are on the table sinces many
> years.
> In order to convince the community to put efforts (time and money) to
> release their data respecting such standard approach we need to illustrate
> the benefit of doing so in the largest possible ways.
>
> Displaying the data into a client (e.g.. The JavaScript API) is a absolutely
> needed step. But despite the fact the default API based access this not a
> per say SOS standard service access (I know it is possible of doing so in
> the Client), this is definitely not enough for sustaining a speech about the
> ubiquity of the service oriented architecture approach.
>
> I know there are other way to illustrate this and I've seen (not tested) the
> ArcGIS plugin that tho in that direction (use of standard) even though this
> is not based on a free and Open Source platform.
>
> A lot of our targeted practitioners for Renewable Energy applications use
> Open Source tools such a QGIS for integrating resources such as maps,
> layers, and of course in-situ measurements (currently using local flat file)
> for their own business or research. Therefore having a working plugin for
> one of the main Open source GIS platform could definitely leverage the
> interest of providing such effort to enable access to service based in-situ
> measurement. 52N solution being one the highly ranked choice for supporting
> the SOS side.
>
> I've already discussed with some in-situ data integrator regarding SOS
> in-situ access into QGIS and they were on the same line as I do.
>
> I do not precisely know the usage of such mix between map/layers based
> information and in-situ measurements around the community that use 52N
> solution, but I would be delighted to ear from you about this.
>
> Do you have such need ?
> Has any of you tested the mentioned plugin ?
> Are you successfully using the ArcGIS one ?
> Do you have any knowledge or idea to make this plugin works for 52N SOS
> resources ?
>
> Thanks for reading and for your further comments.
>
> Cheers,
>
> Lionel
>
>
>
> --
> View this message in context: http://sensorweb.forum.52north.org/QGIS-SOS-Plugin-tp4028298p4028314.html
> Sent from the 52° North - Sensor Web Community Forum mailing list archive at Nabble.com.
> _______________________________________________
> SWE mailing list
> [hidden email]
> http://list.52north.org/mailman/listinfo/swe
> http://sensorweb.forum.52north.org
> Please respect our mailing list guidelines:
> http://52north.org/resources/mailing-lists-and-forums/guidelines
>

_______________________________________________
SWE mailing list
[hidden email]
http://list.52north.org/mailman/listinfo/swe
http://sensorweb.forum.52north.org
Please respect our mailing list guidelines:
http://52north.org/resources/mailing-lists-and-forums/guidelines
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: QGIS SOS Plugin

Lionel Menard
Hi Just,
let me first wish you and all the forum participants a happy new year 2016.

Thanks for your feedback. Yes you're right about open source development under a graduation project form. I've been maybe fooled by the screenshot available in the doc associated with the plugin as it looked like an already final working product.

The SOS performance you underline is also an issue we've reported using the 52N JavaScript client.

In our case a user might be interested to display maps of solar radiation over a region/country and be aware that at some location (to be displayed as a query able layer) there some additional local measurements available to be spotted, displayed (tab, graph) and downloaded (csv). All of this based on SOS Search and Discovery.

If you have any news on this topic, please shared them with the group.

Thanks once again.

All the best.

Lionel

   
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

ArcGIS SOS Extensions

Scholten, Daniel
Dear Lionel and Just,
Dear all,

we want to navigate/explore timeseries data and to download/export it through ArcMap. As a first step, we focus on functionality similar to what the 52N-WebClient offers, but integrated into ArcMap. In particular we want to show navigable graphs inside of ArcMap. That means that at least for now we skip the many ideas one can get when combining information about time AND location.

As far as I know the currently existing ArcGIS-Extensions cannot do that. I would be interested in any experience with ArcGIS-SOS-Extensions.

We are aware that ArcGIS Desktop will be replaced by ArcGIS Pro in the future, but decided to invest in an extension for ArcGIS Desktop, since we estimate for at least five years to stay with ArcGIS Desktop. We still are preparing/ planning the project and are very interested in collaborating with others. Please let me know if you have similar interests in extending ArcMap.

What is our domain and application? We manage a river, do flood control and run wastewater treatment plants. Thus we have hydrological data as well as water quality and energy consumption monitoring. We have measurements and also simulation data. Right now we want to distribute data only inside our organization.

Best wishes
Daniel Scholten
__________________________________________________
Daniel Scholten

Niersverband
Stabsstelle Informations- und Modelltechnik (IMT)
Sachbereich Softwaretechnik
Am Niersverband 10
41747 Viersen

Tel.: +49 2162 3704 - 460 [(Sekretariat: - 101)]
Fax: +49 2162 3704 - 444
E-Mail: [hidden email]

Körperschaft des öffentlichen Rechts, Sitz: Viersen
Vorsitzender des Verbandsrates: Rolf A. Königs, Vorstand: Prof. Dr.-Ing. Dietmar Schitthelm



-----Ursprüngliche Nachricht-----
Von: SWE [mailto:[hidden email]] Im Auftrag von Lionel Menard
Gesendet: Dienstag, 5. Januar 2016 09:29
An: [hidden email]
Betreff: Re: [52N SWE] QGIS SOS Plugin

Hi Just,
let me first wish you and all the forum participants a happy new year 2016.

Thanks for your feedback. Yes you're right about open source development under a graduation project form. I've been maybe fooled by the screenshot available in the doc associated with the plugin as it looked like an already final working product.

The SOS performance you underline is also an issue we've reported using the 52N JavaScript client.

In our case a user might be interested to display maps of solar radiation over a region/country and be aware that at some location (to be displayed as a query able layer) there some additional local measurements available to be spotted, displayed (tab, graph) and downloaded (csv). All of this based on SOS Search and Discovery.

If you have any news on this topic, please shared them with the group.

Thanks once again.

All the best.

Lionel

   



--
View this message in context: http://sensorweb.forum.52north.org/QGIS-SOS-Plugin-tp4028298p4028324.html
Sent from the 52° North - Sensor Web Community Forum mailing list archive at Nabble.com.
_______________________________________________
SWE mailing list
[hidden email]
http://list.52north.org/mailman/listinfo/swe
http://sensorweb.forum.52north.org
Please respect our mailing list guidelines:
http://52north.org/resources/mailing-lists-and-forums/guidelines
_______________________________________________
SWE mailing list
[hidden email]
http://list.52north.org/mailman/listinfo/swe
http://sensorweb.forum.52north.org
Please respect our mailing list guidelines:
http://52north.org/resources/mailing-lists-and-forums/guidelines
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: QGIS SOS Plugin ! NEWS !

Lionel Menard
In reply to this post by Just van den Broecke-2
Dear colleagues,
still time to re-open this thread by wishing you all a happy new year 2017.
Last post was a year ago from now. I did not see in the forum any new post related to this topic. Good news is that things might move a ahead as we are going to have a student that will be fully dedicated on this topic for a 6 month training period that will be part of his curriculum.

Starting on February 1st I expect that we could be able to tackle access, display and download of OGC/SOS based resources into QGIS desktop application.

I intend to broadly open suggestions and guidances from you during this 6 month period. In case we could not be successful to release a fully working plugin to the SOS/QGIS community it would be very helpful to pave the way as properly as possible for other to keep on working on this topic. In case we did, having you aboard will definitively speed up the process and consolidate the plugin with possible testing phases.

So I will ask our student to interact with you on this forum and hope you'll be supporting him with your advices, experience and incentive...

You'll will hear from us in the coming days/weeks....

Thanks again in advance for your support.

All the best,

Lionel
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: QGIS SOS Plugin

Scholten, Daniel
Dear Lionel,

I think one main issue will be the small differences between different SOS-provider-software, for example the implementation of a GetDataAvailability-operation vs. the use of a GetCapabilities-contents-section with one timeseries per offering. For a webportal we want to make ready for SOS we plan to solve this issue by using XSLT-definitions so that for different SOS the way to collect all necessary meta-information can be stored as xslt.

Also be aware of the fact that while the SOS-specification is quite comprehensive regarding metadata, your SOS-provider-software may implement only a small subset. Working with timeseries data in a GIS (in our use cases) is half the fun with small metadata at hand.

Performance may also be an issue as we discussed when we met during the SWE-conference in Münster 2016 (or 2015?), if you still plan to use it with bigger amounts of data.

Please let us know if you have particular questions or something to try out.

Have a nice day!
Daniel
__________________________________________________
Daniel Scholten

Niersverband
Stabsstelle Informations- und Modelltechnik (IMT)
Sachbereich Softwaretechnik
Am Niersverband 10
41747 Viersen

Tel.: +49 2162 3704 - 460 [(Sekretariat: - 101)]
Fax: +49 2162 3704 - 444
E-Mail: [hidden email]

Körperschaft des öffentlichen Rechts, Sitz: Viersen
Vorsitzender des Verbandsrates: Rolf A. Königs, Vorstand: Prof. Dr.-Ing. Dietmar Schitthelm


-----Ursprüngliche Nachricht-----
Von: SWE [mailto:[hidden email]] Im Auftrag von Lionel Menard
Gesendet: Montag, 30. Januar 2017 17:10
An: [hidden email]
Betreff: Re: [52N SWE] QGIS SOS Plugin ! NEWS !

Dear colleagues,
still time to re-open this thread by wishing you all a happy new year 2017.
Last post was a year ago from now. I did not see in the forum any new post related to this topic. Good news is that things might move a ahead as we are going to have a student that will be fully dedicated on this topic for a 6 month training period that will be part of his curriculum.

Starting on February 1st I expect that we could be able to tackle access, display and download of OGC/SOS based resources into QGIS desktop application.

I intend to broadly open suggestions and guidances from you during this 6 month period. In case we could not be successful to release a fully working plugin to the SOS/QGIS community it would be very helpful to pave the way as properly as possible for other to keep on working on this topic. In case we did, having you aboard will definitively speed up the process and consolidate the plugin with possible testing phases.

So I will ask our student to interact with you on this forum and hope you'll be supporting him with your advices, experience and incentive...

You'll will hear from us in the coming days/weeks....

Thanks again in advance for your support.

All the best,

Lionel



--
View this message in context: http://sensorweb.forum.52north.org/QGIS-SOS-Plugin-tp4028298p4028817.html
Sent from the 52° North - Sensor Web Community Forum mailing list archive at Nabble.com.
_______________________________________________
SWE mailing list
[hidden email]
http://list.52north.org/mailman/listinfo/swe
http://sensorweb.forum.52north.org
Please respect our mailing list guidelines:
http://52north.org/resources/mailing-lists-and-forums/guidelines
_______________________________________________
SWE mailing list
[hidden email]
http://list.52north.org/mailman/listinfo/swe
http://sensorweb.forum.52north.org
Please respect our mailing list guidelines:
http://52north.org/resources/mailing-lists-and-forums/guidelines
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: QGIS SOS Plugin ! NEWS !

Alexandre Barbusse
In reply to this post by Lionel Menard
Hi everyone!

I am the student mentioned by Lionel above. The purpose of my final year traineeship - for a six months period which will end on July 31 - is to contribute to this plugin development.

Here are some news about the development.
For now, I am just starting working with the QGIS environment on a Qt-based user interface. Before, I mostly focused on how to bring the necessary tools using a Python script which follows the plugin workflow requirements.
To do so, I decided to use the OWSLib package (https://geopython.github.io/OWSLib/) which makes parsing steps easier.
Sadly, only O&M observations from SOS 2.0 GetObservation responses can be extracted using this library, at this time.
Therefore, we have decided to work with SOS 2.0. But OWSLib is an Open Source package, so SOS 1.0 add-ons could follow.

Here is a first draft of the plugin's workflow, with corresponding Python tools or operations available in parenthesis:

1. Connect to SOS server using SOS service url (string object in Python)
2. Display a general description of the service using GetCapabilities request
3. Display stations with SOS bounding box (list of tuples (bounding box) with 4 coordinates (left, bottom, right, top) in WGS84)
4. Select station (select tuple from list)
5. Display SOS offerings for selected station (list of OWSLib offerings objects)
6. Select offering (from previous list)
7. Display SOS ObservedProperties for selected offering (list of OWSLib ObservedProperties objects)
8. Select ObservedProperty (from previous list)
9. Select TimePeriod (ISO-format datetime.datetime starting time/ending time)
10. GetObservation (OWSLib GetObservation request function)
11. Get time series (list of OWSLib MeasurementObservations)
12. Provide array and plot visualizations buttons (Python Matplotlib plots for now)
13. Provide a “download time series” button (csv export)

We tested a Python script which follows this workflow on a couple of SOS servers, with priority given to 52north implementation and webservice-energy SOS server (http://insitu.webservice-energy.org/52n-sos-webapp/sos?service=SOS&request=GetCapabilities).

We benchmarked processing times for different time-steps and time periods of a GetObservation from this SOS server request using the OWSLib package in Python.
Performance could be an issue when retrieving measurements which have been taken every minute:

   It takes around 10-15s to get one day of measurements every minute;
   It takes around 2 minutes to get one week of measurements every minute;
   It takes no less than 10 minutes to get one month of measurements every minute, which is far too slow in the context of the plugin use.

Despite the poor performance of the process, we have decided to move forward, putting this issue aside as the ui design is independent from it.

Now I’m looking forward to getting your reflections and advice on this first step of development.
Any feedback is more than welcome!


All best,

Alexandre
Loading...