Add database option

[7neyugn7]

Add an option to store messages in database. Check performance.

[7neyugn7]

All the server needs to store is messages being transferred between clients (sources and sinks). Considering between RDMS Postgres | MongoDB | Redis.

http://slashdot.org/topic/bi/sql-vs-nosql-which-is-better/
http://redis.io/

From preliminary research, looks like non-SQL is better since since they store data as native JS objects or JSON.
Options:
MongoDB
Redis
Couch
Riak

[7neyugn7]

MongoDB seems to be a better option to deal with large datasets. MongoDB it is then.

[7neyugn7]

to enable database recording:

node app.js 9090 / true

[7neyugn7]

A better way needed to organise between runs. i.e so users can view results from previous runs as well. Not just the current one.

[7neyugn7]

7neyugn7/IOPortlet#1

[7neyugn7]

From the portal, only one instance of a workflow can be executed at one time. But the portal does not keep track of the runs. I need to store information about runs if want to access the history.

From the portal, it needs to record:

• run id
• workflow id
• start run
• end run
• maybe provenance database

In the server, it needs to record:

• actor definition - so that the history portlet can get the appropriate objects to display data
• run id, and time
• workflow id
• messages exchanged between IOActors, IOPortlets. The history porlet will just display messages from IOActors.

In the portlet, it needs to record:

• history of the messages

Now the question is whether the database should be centralised ?

[7neyugn7]

Note that enabling viewing history of the current run is easy. But viewing history of previous runs are harder. Keep in mind for the future extension.

[7neyugn7]

Possible Solution:

• in each workflow, three parameters are required: "username", "password" and "runid". username and password are string parameters
• when the workflow is executed, these two parameters are passed to workflow
• IOActor will look for these two parametrs in the top level container, if not --> throw exception
• add these two parameters in the definition

Changes needed:

• WorkflowsList portlet
• Slavisa kepler client
• IOActor
• app.js to include run id, etc.