Java Client

The Java client of NebulaStream allows provide a Java interface to build and submit queries to NebulaStream.

Connecting to a NES Instance

NebulaStreamRuntime nebulaStreamRuntime = NebulaStreamRuntime.getRuntime();
nebulaStreamRuntime.getConfig().setHost("localhost")
               .setPort("8081");

Submitting a Query

LogicalStream ysbLogicalStream = ner.getLogicalStream("ysb");

Query query = new Query().from(ysbLogicalStream)
                .filter(Predicate.Attribute("user_id").greaterThanOrEqual(1).and(Predicate.Attribute("user_id").lessthanThan(10000)))
                .map("d5", Expression.multiply("d3","d4"))
                .print();
                
int queryId = nebulaStreamRuntime.executeQuery(query.generateCppCode(), "BottomUp");

Supported Operators

How to use the Filter Operator

Query filterQuery = new Query().from(ysbLogicalStream)
                .filter(predicate);

A predicate can be build with the following methods:

Predicate.Attribute("fieldname") //followed by one of those filter methods:
   .equal(intValue)
   .greaterThan(intValue)
   .greaterThanOrEqual(intValue)
   .lessThan(intValue)
   .lessThanOrEqual(intValue)
   
    //You can combine multiple predicates with:
   .and(predicate)
   .or(predicate)

How to use the Map Operator

   Query mapQuery = new Query().from("defaultLogical").map("value1", Expression.add("value1","value2"));

First the .map() method has to be used. As a parameter a fieldName and Expression has to be hand over.

You can use the following Expressions for the mapping:

.constant(Intvalue)
.add("value1","value2")
.substract("value1","value2")
.multiply("value1","value2")
.divide("value1","value2")

How to use the Window Operator

Query tumblingWindowQuery = new Query().from("exdra")
                .window(TumblingWindow.of(new EventTime("metadata_generated"), TimeMeasure.milliseconds(10)))
                .apply(Aggregation.sum("features_properties_capacity"));

Query slidingWindowQuery = new Query().from("exdra")
                .window(SlidingWindow.of(new EventTime("metadata_generated"), TimeMeasure.milliseconds(10), TimeMeasure.seconds(30)))
                .apply(Aggregation.sum("features_properties_capacity"));

First the .window() method has to be used. As parameters it takes a WindowDefinition. To define the Aggregation use the apply method.

The WindowDefinition can be build with two different classes:

TumblingWindow.of(TimeCharacteristic, TimeMeasure) //where the TimeCharacteristic gets implemented by EventTime("fieldName") and the TimeMeasure by: 
   TimeMeasure.milliseconds(intValue)
   TimeMeasure.seconds(intValue)
   TimeMeasure.minutes(intValue)
   TimeMeasure.hours(intValue)

SlidingWindow.of(TimeCharacteristic, size, slide) //where size and slide are TimeMeasure objects as above.

For the Aggregation is currently just the .sum("fieldName") implemented, as in the examples shown. It is also possible to define a target column for the aggregation result with the .as("columnName") method with the columnName as the parameter.

How to use the Window by Key Operator

The .byKey() method is used after the window operator, before the aggregation.

Query tumblingWindowByKeyQuery = new Query().from("exdra")
                .window(TumblingWindow.of(new EventTime("metadata_generated"), TimeMeasure.milliseconds(10)))
                .byKey("id")
                .apply(Aggregation.sum("features_properties_capacity").as("sum_features_properties_capacity"));

How to use the Join With Operator

The .joinWith() method needs two queries to join.
The first query calls the method with the second as its parameter.
Furthermore the leftFieldName and rightFieldName to join on, has to be defined with .where(leftFieldName) and .equalsTo(rightFieldName) methods.
At last a WindowDefinition, as defined above with the WindowOperators has to be hand over via the window(WindowDefinition) method.

Query q1 = new Query();
q1.from("car");
        
Query q2 = new Query();
q2.from("truck");

// join q1 and q2 on q1.id_car == q2.id_truck with tumbling window of 10ms on timestamp attribute
q1.joinWith(q2)
    .where("id_car")
    .equalsTo("id_truck")
    .window(TumblingWindow.of(new EventTime("timestamp"), TimeMeasure.milliseconds(10)));

How to use the Union With Operator

The .unionWith() method needs two queries to unite. The first query calls the method with the second as its parameter.

Query q1 = new Query();
q1.from("car");

Query q2 = new Query();
q2.from("truck");

// union q1 and q2
q1.unionWith(q2);

How to use the Sink Operator

There are different Sink Operators, which can be used:

   // create a Query with PrintSink()
   Query q1 = new Query();
   q1.from("exdra").sink(new PrintSink());

   // create a Query with ZMQSink Parameters: IP, Port
   Query q2 = new Query();
   q2.from("exdra").sink(new ZMQSink("localhost", 1234));
   
   // create a Query with FileSink Parameters: Filepath, Textformat, Mode
   Query q4 = new Query();
   q4.from("exdra").sink(new FileSink("/tmp/out.txt", "CSV", "OVERRIDE"));
   
   // create a Query with NullOutputSink
   Query q5 = new Query();
   q5.from("exdra").sink(new NullOutputSink());

Topology Management

To get the topology as a Graph<TopologyEntry, TopologyLink> Object, you can use this method

Graph<TopologyEntry, TopologyLink> topology = nebulaStreamRuntime.getNesTopology();

Query Management

Query Catalog

To get the whole Query catalog as a ArrayList<QueryCatalogEntry>, you can use this method.

ArrayList<QueryCatalogEntry> catalog = nebulaStreamRuntime.getAllRegisteredQueries();

Query Catalog filtered by status

If you just want the queries with a specific status, you can use this method.
The available statuses are: Registered, Scheduling, Running and Failed.

ArrayList<QueryCatalogEntry> catalogRegistered = nebulaStreamRuntime.getRegisteredQueryWithStatus("Registered");

Getting the Query Plan

To get the query plan of a specific query as a Graph<LogicalQuery, DefaultEdge>, you can use this method.
The query id has to be hand over to specify.

Graph<LogicalQuery, DefaultEdge> queryPlan = nebulaStreamRuntime.getQueryPlan(queryId);

Getting the Execution Plan

To get the excecution plan of a specific query as a Graph<ExecutionNode, ExecutionLink>, you can use this method.
The query id has to be hand over to specify.

Graph<ExecutionNode, ExecutionLink> executionPlanGraph = nebulaStreamRuntime.getExecutionPlan(queryId);

Deleting a Query

To delete a query, you can use this method.
It returns true, if it was successfully deleted.

boolean deleted = nebulaStreamRuntime.deleteQuery(queryId);

Stream Management

Logical Streamcatalog

To get the logical streamcatalog as a List<String>, you can use this method.

List<String> StreamCatalog = nebulaStreamRuntime.getAvailableLogicalStreams();

Adding a Logical Stream

To add a logical stream,you can use this method.
The streamname and the schema(in C++) has to be submitted as parameters.
It returns true, if it was successfully added.

String streamname = "TestLogiStream";
String schema = "Schema::create()->addField(\"test\",INT32);";

boolean added = nebulaStreamRuntime.addLogicalStream(streamname, schema);

Updating a Logical Stream

To update a logical stream,you can use this method.
The streamname to specify the stream and the updated schema(in C++) has to be submitted as parameters.
It returns true, if it was successfully updated.

String streamname = "TestLogiStream";
String schema = "Schema::create()->addField(\"anothertest\",INT32);";

boolean updated = nebulaStreamRuntime.updateLogicalStream(streamname, schema);

Deleting a Logical Stream

To delete a logical stream,you can use this method.
The streamname to specify the stream has to be submitted as a parameter.
It returns true, if it was successfully deleted.

String streamname = "TestLogiStream";

boolean deleted = nebulaStreamRuntime.deleteLogicalStream(streamname);

Physical Streamcatalog

To get all physical streams of a logical stream as a ArrayList<String>, you can use this method.
To specify the logical stream, the logical streamname has to be submitted as a parameter.

ArrayList<String> physicalStreamOfDefaultLogical = nebulaStreamRuntime.getAvailablePhysicalStreamsOfLogicalStream("default_logical");
 
java_client.txt · Last modified: 2021/12/07 13:35 by 134.96.191.189
 
Recent changes RSS feed Creative Commons License Donate Powered by PHP Valid XHTML 1.0 Valid CSS Driven by DokuWiki