Data source selection and post requests

classic Classic list List threaded Threaded
15 messages Options
Reply | Threaded
Open this post in threaded view
|

Data source selection and post requests

Andi1979
Hello ... I am new to the 52north SOS .
I follow the instructions and deploy the system on my Tomcat server.
I have some questions:
1- after deploying the SOS, on the window to select the data source, what is the difference between the options of the same database type , for example :
 postgresql
Postgresql core
Postgresql custom

2- how can I add / remove a data source option ?
3-How to add a new request example? For example I would add a new getcapabilities or getobservation request ?

Hope my questions are clear
Thank u
Andi
Reply | Threaded
Open this post in threaded view
|

Re: Data source selection and post requests

Carsten Hollmann
Hi Andi

> 1- after deploying the SOS, on the window to select the data source, what is
> the difference between the options of the same database type , for example :
> postgresql

This datasource should be selected if you would use the OGC SOS 2.0
transactional operations (InsertSensor, InsertObservatiob, ...). This
creates/uses the full 52N SOS database model.

> Postgresql core

The "Postgresql Core" datasource uses/creates only the mandatory
database schema to provide observation data and metadata.

> Postgresql custom

This should be used if you have modified the Hibernate mapping files to
provide observation data from a custom database. This datasource does
not check if the database schema corresponds to the schema defined in
the mapping files because customized mapping files may contain
subselects and formulas.

> 2- how can I add / remove a data source option ?

It depends on which option should be removed. If all datasources for a
DBMS should be removed, delete the lib from the deployed SOS.

To add a data source option you have to implement the
org.n52.sos.ds.Datasource interface, define the datasource in the
org.n52.sos.ds.Datasource ServiceLoader file, build the JAR and add the
JAR to the SOS.

https://wiki.52north.org/SensorWeb/SensorObservationServiceDeveloperGuide#Datasources

> 3-How to add a new request example? For example I would add a new
> getcapabilities or getobservation request ?

For XML requests you have to create a XML file that contains the
request. Then link the file in the client-config.json file
(../static/conf folder of the deployed SOS) equal to the other XML
examples.
For KVP or JSON request add the request to client-config.json file as
the existing requests.

Best,
Carsten
_______________________________________________
SWE mailing list
[hidden email]
http://list.52north.org/mailman/listinfo/swe
http://sensorweb.forum.52north.org
Please respect our mailing list guidelines:
http://52north.org/resources/mailing-lists-and-forums/guidelines
Reply | Threaded
Open this post in threaded view
|

Re: Data source selection and post requests

Andi1979
Hello, 
Thank you Carsten for your reply..
I deploy the 52Noth SOS to my tomcat, the postgresql service is working .. 
using the pgadmin, i build a postgresql database (similar to default 52north data model), and fill some random data..During the SOS configuration, I change the database name (by defualt : SOS) to "TestDB". 

I tried to apply some Getobservation request, I just modify the exsisting test example : 

<?xml version="1.0" encoding="UTF-8"?>
<sos:GetObservation
    xmlns:sos="http://www.opengis.net/sos/2.0"
    xmlns:fes="http://www.opengis.net/fes/2.0"
    xmlns:gml="http://www.opengis.net/gml/3.2"
    xmlns:swe="http://www.opengis.net/swe/2.0"
    xmlns:xlink="http://www.w3.org/1999/xlink"
    xmlns:swes="http://www.opengis.net/swes/2.0"
    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" service="SOS" version="2.0.0" xsi:schemaLocation="http://www.opengis.net/sos/2.0http://schemas.opengis.net/sos/2.0/sos.xsd">
  
    <sos:procedure>http://www.52north.org/test/procedure/mysensor</sos:procedure>
  
    <sos:observedProperty>http://www.52north.org/test/observableProperty/prop1</sos:observedProperty>
 
    <sos:featureOfInterest>http://www.52north.org/test/featureOfInterest/house1</sos:featureOfInterest>
    
   
    <sos:responseFormat>http://www.opengis.net/om/2.0</sos:responseFormat>
</sos:GetObservation>

I get this Response : 

200 OK
Date: Tue, 04 Oct 2016 13:13:27 GMT
Content-Encoding: gzip
Server: Apache-Coyote/1.1
Transfer-Encoding: chunked
Content-Type: application/xml;charset=UTF-8

<?xml version="1.0" encoding="UTF-8"?>
<ows:ExceptionReport xmlns:ows="http://www.opengis.net/ows/1.1" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" version="2.0.0" xsi:schemaLocation="http://www.opengis.net/ows/1.1http://schemas.opengis.net/ows/1.1.0/owsAll.xsd">
  <ows:Exception exceptionCode="InvalidParameterValue" locator="observedProperty">
    <ows:ExceptionText>The value 'http://www.52north.org/test/observableProperty/prop1' of the parameter 'observedProperty' is invalid</ows:ExceptionText>
  </ows:Exception>
  <ows:Exception exceptionCode="InvalidParameterValue" locator="procedure">
    <ows:ExceptionText>The value 'http://www.52north.org/test/procedure/mysensor' of the parameter 'procedure' is invalid</ows:ExceptionText>
  </ows:Exception>
  <ows:Exception exceptionCode="InvalidParameterValue" locator="featureOfInterest">
    <ows:ExceptionText>The value 'http://www.52north.org/test/featureOfInterest/house1' of the parameter 'featureOfInterest' is invalid</ows:ExceptionText>
  </ows:Exception>
</ows:ExceptionReport>

so basiclly, i think the SOS can not access to the data inside my data , becauss, for example inside my database, i have the procedure table whith the data value "mysensor"


I do not know what i miss here ?  shjould i connect my databse in another way ? i just put the database name and the port number .. 

Thank you
Andi 

Reply | Threaded
Open this post in threaded view
|

Re: Data source selection and post requests

Carsten Hollmann
Hi Andi,

How did you insert the data into the SOS?
With the transactional operations?

If not, then necessary information may be missing in some tables.

Have you checked the Capabilites of the SOS (GetCapabilities request)?

Does the response contain the values from your TestDB in the contents
section?

Best,
Carsten
_______________________________________________
SWE mailing list
[hidden email]
http://list.52north.org/mailman/listinfo/swe
http://sensorweb.forum.52north.org
Please respect our mailing list guidelines:
http://52north.org/resources/mailing-lists-and-forums/guidelines
Reply | Threaded
Open this post in threaded view
|

Re: Data source selection and post requests

Andi1979
Hello Carsten,
I insert the data to the postgresql database immedialty using the pgadmin interface.

I sent describesensor request and the response shows that the parameter 'procedure' is invalid, the response error samiliar to the error massage of getobservation request which sent at the last email ..


Also I sent a getcapabilities request, this is the response,
Response.txt

I can see that my data is not there ..

Best,
Andi
Reply | Threaded
Open this post in threaded view
|

Re: Data source selection and post requests

Andi1979
In reply to this post by Carsten Hollmann
Hello Carsten, 


I am traying to add a new data source option on the main deploymnet screen,  I did the following: 
*I created a submodule called "Mydatasource" ,
* i implement a new data source class (public class Mydatasource implements Datasource)... Also i implemented the  DataConnectionProvider class 
*inside the main pom.xml,, I added the module <module>Mydatatsource </module> 

*the mydatasource module pom.xml
 
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd">

<parent>
    <groupId>org.n52.sensorweb.sos</groupId>
    <artifactId>52n-sos</artifactId>
    <version>4.0.0</version>
    </parent>

    <modelVersion>4.0.0</modelVersion>
    <artifactId>MyDatasource</artifactId>
    <packaging>jar</packaging>
	<name>MyDatasource</name>
  
    <dependencies>
        <dependency>
            <groupId>${project.groupId}</groupId>
            <artifactId>api</artifactId>
        </dependency>


*Inside the file  : "/52n-sos/Mydatasouce/target/classes/META-INF/services/org.n52.sos.ds.Datasource" , I added the line " org.n52.sos.ds.datasource.Mydatasouce"

Then i rebuild the project with no errors, but after I deploy it to tomcat, i can not find mydatasource option on the drop-down list !! 

I woud be glade to know what i miss here 

Thank you in advance 

Andi 

Reply | Threaded
Open this post in threaded view
|

Re: Data source selection and post requests

Andi1979
Hi carsten,

I added jar dependency to the main Pom.xml not only the module name, I do not know if that is enough , but now I can find the Mydatasource option between the drop-down list ..

But .. when I select the option, the connection details are not appear (url, username, .....) even I define them on the data source implementation.

The datasource implementation as the following:

package org.n52.sos.ds.datasource;

import java.io.IOException;
import java.sql.DriverManager;
import java.sql.Connection;
import java.sql.SQLException;
import java.sql.Statement;



import java.util.Map;
import java.util.Properties;
import java.util.Set;

import org.n52.sos.config.SettingDefinition;
import org.n52.sos.config.settings.StringSettingDefinition;
import org.n52.sos.ds.Datasource;
import org.n52.sos.ds.DatasourceCallback;

import com.google.common.collect.Sets;

public class Mydatasource implements Datasource {

private static final String DIALECT_NAME = "XXX";
       protected static final String USERNAME_DESCRIPTION =" default is \"XXX\".";
    protected static final String USERNAME_DEFAULT_VALUE = "XXX";
    protected static final String PASSWORD_DESCRIPTION = "\"PASSWORD\".";

    protected static final String PASSWORD_DEFAULT_VALUE = "PASSWORD";
  private static final String KEY_XXX_URI = "uri.path.XXX";
  public  static final String XXX_URL="jdbc:XXX://localhost/";
  protected static final String XXX_DRIVER_CLASS = "org.XXX.jdbc.Driver";  
 
      protected static final String HOST_DESCRIPTION = "defaulttt \"localhost\".";

    protected static final String PORT_DESCRIPTION = " 0000";
 

     static final int PORT_DEFAULT_VALUE = 0000;

  

//**********
 
public static void main(String [] args) throws IOException
 {
   try
   {
     Class.forName(XXX_DRIVER_CLASS);
   }
   catch (ClassNotFoundException e)
   {
     System.out.println("Driver is not in the CLASSPATH -> " + e);
   }

   try
   {
     Connection conn = DriverManager.getConnection(XXX_URL);
     Statement st = conn.createStatement();
           System.out.println("=====");
        

       }
   catch (SQLException e)
   {
     System.out.println(e);
   }
 System.exit(0);
 }
 
//***********
 
 
protected final StringSettingDefinition createXXXUri() {
return new StringSettingDefinition().setGroup(BASE_GROUP).setOrder(1)
.setKey(KEY_XXX_URI).setTitle("Path to XXX-Directory")
.setDefaultValue(DEFAULT_XXX_URI);
}

@Override
public final String getDialectName() {
return DIALECT_NAME;
}

@Override
public final Set<SettingDefinition<?, ?>> getSettingDefinitions() {
return Sets.<SettingDefinition<?, ?>> newHashSet(createXXXUri(),
createXXXUri());

}

@Override
public final Set<SettingDefinition<?, ?>> getChangableSettingDefinitions(
final Properties arg0) {
return Sets.<SettingDefinition<?, ?>> newHashSet(createXXXUri(),
createXXXUri());
}

@Override
public final boolean checkIfSchemaExists(final Map<String, Object> arg0) {
// TODO Auto-generated method stub
return false;
}

@Override
public final boolean checkIfSchemaExists(final Properties arg0,
final Map<String, Object> arg1) {
// TODO Auto-generated method stub
return false;
}

@Override
public final boolean checkSchemaCreation(final Map<String, Object> arg0) {
// TODO Auto-generated method stub
return false;
}

@Override
public void clear(final Properties arg0) {
// TODO Auto-generated method stub

}

@Override
public final String[] createSchema(final Map<String, Object> arg0) {
// TODO Auto-generated method stub
return new String[0];
}

@Override
public final String[] dropSchema(final Map<String, Object> arg0) {
// TODO Auto-generated method stub
return new String[0];
}

@Override
public void execute(final String[] arg0, final Map<String, Object> arg1) {
// TODO Auto-generated method stub

}

@Override
public final Properties getDatasourceProperties(
final Map<String, Object> settings) {
Properties p = new Properties();
p.put(KEY_XXX_URI, settings.get(KEY_XXX_URI));
return p;
}

@Override
public final Properties getDatasourceProperties(final Properties arg0,
final Map<String, Object> settings) {
// TODO Auto-generated method stub
return getDatasourceProperties(settings);
}

@Override
public final boolean needsSchema() {
// TODO Auto-generated method stub
return false;
}

@Override
public final boolean supportsClear() {
// TODO Auto-generated method stub
return false;
}

@Override
public void validateConnection(final Map<String, Object> arg0) {
// TODO Auto-generated method stub

}

@Override
public void validateConnection(final Properties arg0,
final Map<String, Object> arg1) {
// TODO Auto-generated method stub

}

@Override
public void validatePrerequisites(final Map<String, Object> arg0) {
// TODO Auto-generated method stub

}

@Override
public void validatePrerequisites(final Properties arg0,
final Map<String, Object> arg1) {
// TODO Auto-generated method stub

}

@Override
public void validateSchema(final Map<String, Object> arg0) {
// TODO Auto-generated method stub

}

@Override
public void validateSchema(final Properties arg0,
final Map<String, Object> arg1) {
// TODO Auto-generated method stub

}

    @Override
    public String[] updateSchema(Map<String, Object> arg0) {
        // TODO Auto-generated method stub
        return new String[0];
    }

@Override
public String getConnectionProviderIdentifier() {
// TODO Auto-generated method stub
return null;
}

@Override
public String getDatasourceDaoIdentifier() {
// TODO Auto-generated method stub
return null;
}

@Override
public DatasourceCallback getCallback() {
// TODO Auto-generated method stub
return null;
}

@Override
public void prepare(Map<String, Object> settings) {
// TODO Auto-generated method stub
 
}

@Override
public boolean isPostCreateSchema() {
// TODO Auto-generated method stub
return false;
}

@Override
public void executePostCreateSchema(Map<String, Object> databaseSettings) {
// TODO Auto-generated method stub
 
}

@Override
public void checkPostCreation(Properties properties) {
// TODO Auto-generated method stub
 
}

}
Reply | Threaded
Open this post in threaded view
|

Re: Data source selection and post requests

Carsten Hollmann
In reply to this post by Andi1979
Hi Andi,

have you reload the capabilities cache after the insertion?
(via the admin interface) Probably the automatic cache update was not
started.

We recommend to use the SOS transactional operations (InsertSensor,
InsertObservation) to insert data into the database because in that case
all required tables and columns are filled with data.

Best,
Carsten
_______________________________________________
SWE mailing list
[hidden email]
http://list.52north.org/mailman/listinfo/swe
http://sensorweb.forum.52north.org
Please respect our mailing list guidelines:
http://52north.org/resources/mailing-lists-and-forums/guidelines
Reply | Threaded
Open this post in threaded view
|

Re: Data source selection and post requests

Carsten Hollmann
In reply to this post by Andi1979
Hi Andi,

why do you want to define a new datasource?

It would be helpful if you could explain your use case?

Best,
Carsten
_______________________________________________
SWE mailing list
[hidden email]
http://list.52north.org/mailman/listinfo/swe
http://sensorweb.forum.52north.org
Please respect our mailing list guidelines:
http://52north.org/resources/mailing-lists-and-forums/guidelines
Reply | Threaded
Open this post in threaded view
|

Re: Data source selection and post requests

Andi1979
Hello Carsten,
Thank you for your reply.
I am a researcher within a team group.
We have a project under smart cities. This project will use a SOS system (maybe 52north), and store the data in Database.  

My job is to search for the best SOS , so I am traying to compare the 52north SOS with other existing SOS. For the database part, I am not responsible to select the suitable one for the data, but I have to find the availablity to connect the SOS to different Data sources, I can see you already support multiple databases, but I am not sure if my colleges will use one of them for data manipulation. I think they are taking Rasdaman to consideration. The implemented data source is just random, the configuration data will be change according to the selection of the database team.
So it will be helpful if you can help me and let me know why the database configurations are not appear , and what should I do to make the connection working.  😃
Thank you
Andi
Reply | Threaded
Open this post in threaded view
|

Re: Data source selection and post requests

Carsten Hollmann
Hi Andi,

have a look at the existing Datasource implementations as the
PostgresDatasource, etc how it is implemented.

If you would like to use another data source than the existing ones, you
may need to re-implement the "hibernate" module because the implemented
hibernate modules only support databases that are supported by Hibernate
Spatial.

Best,
Carsten
_______________________________________________
SWE mailing list
[hidden email]
http://list.52north.org/mailman/listinfo/swe
http://sensorweb.forum.52north.org
Please respect our mailing list guidelines:
http://52north.org/resources/mailing-lists-and-forums/guidelines
Reply | Threaded
Open this post in threaded view
|

Re: Data source selection and post requests

Andi1979
In reply to this post by Carsten Hollmann
Hello again 😃
I am still traying to make my SOS execute request from my postgres.
I deploy SOS4.x and have a postgres db already filled with data.
The database already have procedure , offering, observedproperty, observation tables (beside many other tables)
I found this configuration tips, which is for SOS3.x :
https://wiki.52north.org/SensorWeb/SosTutorial#Forth_Step:_Testing_the_SOS
I found that I have to configure the services and add my procedures and property names ,by editing dssos.config and SOS.config
I did not find such files on SOS4.x and I 're-read the configuration process from the website , I can not find anything for connecting and configuring my db to SOS .. I do not know what I miss,, I be glad to find a clear steps to make this work ..
Thanks
Andi
Reply | Threaded
Open this post in threaded view
|

Re: Data source selection and post requests

Carsten Hollmann
Hi Andi,

the wiki page you mentioned is for the SOS 3.x as it is defined on the page.

During the installation process you defined the database and the schema
the SOS should use.

If you would like to use your own database schema you have to modify the
Hibernate mapping files or write your own DAOs, if your database model
is completely different from the default SOS database model.

More information about the default database model of the SOS can be
found here:

   https://wiki.52north.org/SensorWeb/SensorObservationServiceDatabaseModel

And about the Hibernate mapping files:

 
https://wiki.52north.org/SensorWeb/SensorObservationServiceHibernateMapping

Best,
Carsten
_______________________________________________
SWE mailing list
[hidden email]
http://list.52north.org/mailman/listinfo/swe
http://sensorweb.forum.52north.org
Please respect our mailing list guidelines:
http://52north.org/resources/mailing-lists-and-forums/guidelines
Reply | Threaded
Open this post in threaded view
|

Re: Data source selection and post requests

Andi1979
Thank you for your reply ..
I already read the links you share, but I will recheck them again. And compare my DB Model to the 52North Model..
Before that, I would ask about the "cache reload" on the admin page.. I press this button and it keeps loading for 14 hours, then I kill the process.. I am using Firefox and I was on developer mode and watching the network tab, there was a request sent every 5-8 millisecond.. but I found 14 hours are very long time..is that normal ???
Reply | Threaded
Open this post in threaded view
|

Re: Data source selection and post requests

Carsten Hollmann
Hi Andi,

I guess the cache reload fails due to some differences between your and
the default database model or some missing data in the tables.
The admin page was not reloaded due to the issue in the cache update, so
the loading was still displayed.

Best,
Carsten
_______________________________________________
SWE mailing list
[hidden email]
http://list.52north.org/mailman/listinfo/swe
http://sensorweb.forum.52north.org
Please respect our mailing list guidelines:
http://52north.org/resources/mailing-lists-and-forums/guidelines