Re: Dec 16 Podling Report

[Suneel Marthi]

Updated the Dec 16 draft, see
https://cwiki.apache.org/confluence/display/ARIATOSCA/Podling+Report+2016-12

If Ok, I'll post the same to Incubator.

Here's my +1

On Sun, Dec 4, 2016 at 9:25 PM, John D. Ament  wrote:

> +1
> Its important to remember that graduation is not around technical merits
> but all about functional.  There are some basic mechanical things - cutting
> valid releases but the bulk of it is around community.
>
> On Sun, Dec 4, 2016 at 7:39 PM Suneel Marthi  wrote:
>
> > What's most important for the graduation is the growth in community that
> > includes committers and contributions from outside of Gigaspaces, that's
> > something we need to work on as a higher priority.
> >
> >
> >
> > --
> >
> > AriaTosca  ARIA TOSCA project offers an easily consumable Software
> > Development Kit(SDK) and a Command Line Interface(CLI) to implement
> > TOSCA(Topology and Orchestration Specification of Cloud Applications)
> > based solutions.  AriaTosca has been incubating since 2016-08-27.
> > Three most important issues to address in the move towards graduation:
> >
> >1. Model storage mechanisem, and execution plugin.
> >
> > >>> Replace this with 'Grow community and enroll new committers"
> >
> >2. Update ARIA website acording to ASF guideline(Include release
> > process for ARIA-TOSCA Project)
> >
> >3. Publish "How To Contribute" guide at ARIA-TOSCA Website.
> >
> >
> >
> >   Any issues that the Incubator PMC (IPMC) or ASF Board wish/need to
> > be aware of?
> > There's an INFRA jira we created over a month ago which has yet to be
> > addressed - please see here
> >
> > INFRA-12733  -
> Ability
> > to create a Sprint Board for AriaTosca WAITING FOR USER
> >
> > It is not entirely critical for the project's progress, but it could
> > be helpful if we indeed get a sprint board and the other things
> > requested in that issue.
> >
> >
> > How has the community developed since the last report?
> >
> >
> >- Open-O is in process for consuming the ARIA ASF package
> >- Cloudfiy is in process for consuming the ARIA ASF package as well.
> >
> >   How has the project developed since the last report?
> >
> >
> >-
> >
> >TOSCA parser migrated into ASF
> >
> >-
> >
> >CI on Appveyor, Jenkins
> >
> >-
> >
> >Sphinx documentation added to the project
> >
> >- CLIs of workflow engine and parser coalesced
> >- TOSCA CSAR packager added
> >- APIs for workflows, operations
> >- Workflow engine task retry support
> >- Workflow engine execution cancel support
> >
> >
> >
> >  Date of last release:
> >   -
> >
> >  When were the last committers or PMC members elected?
> >
> >Project is being established in incubator with the proposed initial
> set
> > of
> >   committers.
> >
> >  Signed-off-by:
> >
> >[ ](ariatosca) Suneel Marthi
> >[ ](ariatosca) John D. Ament
> >[ ](ariatosca) Jakob Homan
> >
> >  Shepherd/Mentor notes:
> >
>

Re: Dec 16 Podling Report

[John D. Ament]

+1
Its important to remember that graduation is not around technical merits
but all about functional.  There are some basic mechanical things - cutting
valid releases but the bulk of it is around community.

On Sun, Dec 4, 2016 at 7:39 PM Suneel Marthi  wrote:

> What's most important for the graduation is the growth in community that
> includes committers and contributions from outside of Gigaspaces, that's
> something we need to work on as a higher priority.
>
>
>
> --
>
> AriaTosca  ARIA TOSCA project offers an easily consumable Software
> Development Kit(SDK) and a Command Line Interface(CLI) to implement
> TOSCA(Topology and Orchestration Specification of Cloud Applications)
> based solutions.  AriaTosca has been incubating since 2016-08-27.
> Three most important issues to address in the move towards graduation:
>
>1. Model storage mechanisem, and execution plugin.
>
> >>> Replace this with 'Grow community and enroll new committers"
>
>2. Update ARIA website acording to ASF guideline(Include release
> process for ARIA-TOSCA Project)
>
>3. Publish "How To Contribute" guide at ARIA-TOSCA Website.
>
>
>
>   Any issues that the Incubator PMC (IPMC) or ASF Board wish/need to
> be aware of?
> There's an INFRA jira we created over a month ago which has yet to be
> addressed - please see here
>
> INFRA-12733  - Ability
> to create a Sprint Board for AriaTosca WAITING FOR USER
>
> It is not entirely critical for the project's progress, but it could
> be helpful if we indeed get a sprint board and the other things
> requested in that issue.
>
>
> How has the community developed since the last report?
>
>
>- Open-O is in process for consuming the ARIA ASF package
>- Cloudfiy is in process for consuming the ARIA ASF package as well.
>
>   How has the project developed since the last report?
>
>
>-
>
>TOSCA parser migrated into ASF
>
>-
>
>CI on Appveyor, Jenkins
>
>-
>
>Sphinx documentation added to the project
>
>- CLIs of workflow engine and parser coalesced
>- TOSCA CSAR packager added
>- APIs for workflows, operations
>- Workflow engine task retry support
>- Workflow engine execution cancel support
>
>
>
>  Date of last release:
>   -
>
>  When were the last committers or PMC members elected?
>
>Project is being established in incubator with the proposed initial set
> of
>   committers.
>
>  Signed-off-by:
>
>[ ](ariatosca) Suneel Marthi
>[ ](ariatosca) John D. Ament
>[ ](ariatosca) Jakob Homan
>
>  Shepherd/Mentor notes:
>

Dec 16 Podling Report

[Suneel Marthi]

What's most important for the graduation is the growth in community that
includes committers and contributions from outside of Gigaspaces, that's
something we need to work on as a higher priority.



--

AriaTosca  ARIA TOSCA project offers an easily consumable Software
Development Kit(SDK) and a Command Line Interface(CLI) to implement
TOSCA(Topology and Orchestration Specification of Cloud Applications)
based solutions.  AriaTosca has been incubating since 2016-08-27.
Three most important issues to address in the move towards graduation:

   1. Model storage mechanisem, and execution plugin.

>>> Replace this with 'Grow community and enroll new committers"

   2. Update ARIA website acording to ASF guideline(Include release
process for ARIA-TOSCA Project)

   3. Publish "How To Contribute" guide at ARIA-TOSCA Website.



  Any issues that the Incubator PMC (IPMC) or ASF Board wish/need to
be aware of?
There's an INFRA jira we created over a month ago which has yet to be
addressed - please see here

INFRA-12733  - Ability
to create a Sprint Board for AriaTosca WAITING FOR USER

It is not entirely critical for the project's progress, but it could
be helpful if we indeed get a sprint board and the other things
requested in that issue.


How has the community developed since the last report?


   - Open-O is in process for consuming the ARIA ASF package
   - Cloudfiy is in process for consuming the ARIA ASF package as well.

  How has the project developed since the last report?


   -

   TOSCA parser migrated into ASF

   -

   CI on Appveyor, Jenkins

   -

   Sphinx documentation added to the project

   - CLIs of workflow engine and parser coalesced
   - TOSCA CSAR packager added
   - APIs for workflows, operations
   - Workflow engine task retry support
   - Workflow engine execution cancel support



 Date of last release:
  -

 When were the last committers or PMC members elected?

   Project is being established in incubator with the proposed initial set of
  committers.

 Signed-off-by:

   [ ](ariatosca) Suneel Marthi
   [ ](ariatosca) John D. Ament
   [ ](ariatosca) Jakob Homan

 Shepherd/Mentor notes:

incubator-ariatosca git commit: code review 2 - wip [Forced Update!]

[mxmrlv]

Repository: incubator-ariatosca
Updated Branches:
  refs/heads/ARIA-30-SQL-based-storage-implementation 9a608318b -> b60199ba5 
(forced update)


code review 2 - wip


Project: http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/repo
Commit: 
http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/commit/b60199ba
Tree: http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/tree/b60199ba
Diff: http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/diff/b60199ba

Branch: refs/heads/ARIA-30-SQL-based-storage-implementation
Commit: b60199ba57c09555782873e4d03313ff661bb13b
Parents: 5ea0b63
Author: mxmrlv 
Authored: Sun Dec 4 18:56:02 2016 +0200
Committer: mxmrlv 
Committed: Sun Dec 4 19:04:38 2016 +0200

--
 aria/__init__.py|   4 +-
 aria/orchestrator/context/toolbelt.py   |   2 +-
 aria/orchestrator/context/workflow.py   |   8 +-
 aria/orchestrator/workflows/core/task.py|  14 +-
 aria/storage/core.py|   8 +-
 aria/storage/models.py  | 195 +--
 aria/storage/sql_mapi.py|  53 +++--
 aria/storage/structures.py  |  34 +---
 tests/mock/context.py   |   6 +-
 tests/mock/models.py|  28 +--
 tests/orchestrator/context/test_workflow.py |   4 +-
 .../orchestrator/workflows/builtin/__init__.py  |   2 -
 .../workflows/builtin/test_execute_operation.py |  14 +-
 .../orchestrator/workflows/builtin/test_heal.py |  16 +-
 .../workflows/builtin/test_install.py   |  11 +-
 .../workflows/builtin/test_uninstall.py |  12 +-
 .../orchestrator/workflows/core/test_engine.py  |   2 +-
 tests/storage/__init__.py   |   4 +-
 tests/storage/test_model_storage.py |  29 +--
 tests/storage/test_models.py|  14 +-
 tests/storage/test_resource_storage.py  |   6 +-
 21 files changed, 229 insertions(+), 237 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/blob/b60199ba/aria/__init__.py
--
diff --git a/aria/__init__.py b/aria/__init__.py
index 5317afa..f5151e3 100644
--- a/aria/__init__.py
+++ b/aria/__init__.py
@@ -56,7 +56,7 @@ def install_aria_extensions():
 del sys.modules[module_name]
 
 
-def application_model_storage(api, api_params=None):
+def application_model_storage(api, api_kwargs=None):
 """
 Initiate model storage for the supplied storage driver
 """
@@ -76,7 +76,7 @@ def application_model_storage(api, api_params=None):
 storage.models.Task,
 ]
 # if api not in _model_storage:
-return storage.ModelStorage(api, items=models, api_params=api_params or {})
+return storage.ModelStorage(api, items=models, api_kwargs=api_kwargs or {})
 
 
 def application_resource_storage(driver):

http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/blob/b60199ba/aria/orchestrator/context/toolbelt.py
--
diff --git a/aria/orchestrator/context/toolbelt.py 
b/aria/orchestrator/context/toolbelt.py
index ceaeb72..ae0e1ff 100644
--- a/aria/orchestrator/context/toolbelt.py
+++ b/aria/orchestrator/context/toolbelt.py
@@ -33,7 +33,7 @@ class NodeToolBelt(object):
 :return:
 """
 assert isinstance(self._op_context, operation.NodeOperationContext)
-filters = {'target_node_instance_fk': 
self._op_context.node_instance.storage_id}
+filters = {'target_node_instance_storage_id': 
self._op_context.node_instance.storage_id}
 for relationship_instance in \
 
self._op_context.model.relationship_instance.iter(filters=filters):
 yield relationship_instance.source_node_instance

http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/blob/b60199ba/aria/orchestrator/context/workflow.py
--
diff --git a/aria/orchestrator/context/workflow.py 
b/aria/orchestrator/context/workflow.py
index 0540590..5ec4044 100644
--- a/aria/orchestrator/context/workflow.py
+++ b/aria/orchestrator/context/workflow.py
@@ -53,8 +53,8 @@ class WorkflowContext(BaseContext):
 now = datetime.utcnow()
 execution = self.model.execution.model_cls(
 id=self._execution_id,
-blueprint_fk=self.blueprint.storage_id,
-deployment_fk=self.deployment.storage_id,
+blueprint_storage_id=self.blueprint.storage_id,
+deployment_storage_id=self.deployment.storage_id,
 workflow_id=self._workflow_id,
 created_at=now,
 status=execution_cls.PENDING,
@@ -69,7 +69,7 @@ class 

[GitHub] incubator-ariatosca pull request #31: ARIA-30-SQL-based-storage-implementati...

[mxmrlv]

Github user mxmrlv commented on a diff in the pull request:

https://github.com/apache/incubator-ariatosca/pull/31#discussion_r90781017
  
--- Diff: aria/storage/sql_mapi.py ---
@@ -0,0 +1,361 @@
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements.  See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License.  You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+"""
+SQLAlchemy based MAPI
+"""
+
+from sqlalchemy.exc import SQLAlchemyError
+from sqlalchemy.sql.elements import Label
+
+from aria.utils.collections import OrderedDict
+
+from aria.storage import (
+api,
+exceptions
+)
+
+
+DEFAULT_SQL_DIALECT = 'sqlite'
+
+
+class SQLAlchemyModelAPI(api.ModelAPI):
+"""
+SQL based MAPI.
+"""
+
+def __init__(self,
+ engine,
+ session,
+ **kwargs):
+super(SQLAlchemyModelAPI, self).__init__(**kwargs)
+self._engine = engine
+self._session = session
+
+def get(self, entry_id, include=None, filters=None, locking=False, 
**kwargs):
+"""Return a single result based on the model class and element ID
+"""
+filters = filters or {'id': entry_id}
+query = self._get_query(include, filters)
+if locking:
+query = query.with_for_update()
+result = query.first()
+
+if not result:
+raise exceptions.StorageError(
+'Requested {0} with ID `{1}` was not found'
--- End diff --

no need anymore


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

incubator-ariatosca git commit: code review 2 - wip

[mxmrlv]

Repository: incubator-ariatosca
Updated Branches:
  refs/heads/ARIA-30-SQL-based-storage-implementation 5ea0b6313 -> 9a608318b


code review 2 - wip


Project: http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/repo
Commit: 
http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/commit/9a608318
Tree: http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/tree/9a608318
Diff: http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/diff/9a608318

Branch: refs/heads/ARIA-30-SQL-based-storage-implementation
Commit: 9a608318bcd249d030e465ca8687b8d059995cc0
Parents: 5ea0b63
Author: mxmrlv 
Authored: Sun Dec 4 18:56:02 2016 +0200
Committer: mxmrlv 
Committed: Sun Dec 4 18:56:02 2016 +0200

--
 aria/__init__.py|   4 +-
 aria/orchestrator/context/toolbelt.py   |   2 +-
 aria/orchestrator/context/workflow.py   |   8 +-
 aria/orchestrator/workflows/core/task.py|  14 +-
 aria/storage/core.py|   8 +-
 aria/storage/models.py  | 185 +--
 aria/storage/sql_mapi.py|  52 +++---
 aria/storage/structures.py  |  34 +---
 tests/mock/context.py   |   6 +-
 tests/mock/models.py|  28 +--
 tests/orchestrator/context/test_workflow.py |   4 +-
 .../orchestrator/workflows/builtin/__init__.py  |   7 +
 .../workflows/builtin/test_execute_operation.py |   4 +-
 .../orchestrator/workflows/builtin/test_heal.py |  10 +-
 .../workflows/builtin/test_install.py   |   9 +-
 .../workflows/builtin/test_uninstall.py |  10 +-
 .../orchestrator/workflows/core/test_engine.py  |   2 +-
 tests/storage/__init__.py   |   4 +-
 tests/storage/test_model_storage.py |  27 +--
 tests/storage/test_models.py|  14 +-
 tests/storage/test_resource_storage.py  |   6 +-
 21 files changed, 201 insertions(+), 237 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/blob/9a608318/aria/__init__.py
--
diff --git a/aria/__init__.py b/aria/__init__.py
index 5317afa..f5151e3 100644
--- a/aria/__init__.py
+++ b/aria/__init__.py
@@ -56,7 +56,7 @@ def install_aria_extensions():
 del sys.modules[module_name]
 
 
-def application_model_storage(api, api_params=None):
+def application_model_storage(api, api_kwargs=None):
 """
 Initiate model storage for the supplied storage driver
 """
@@ -76,7 +76,7 @@ def application_model_storage(api, api_params=None):
 storage.models.Task,
 ]
 # if api not in _model_storage:
-return storage.ModelStorage(api, items=models, api_params=api_params or {})
+return storage.ModelStorage(api, items=models, api_kwargs=api_kwargs or {})
 
 
 def application_resource_storage(driver):

http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/blob/9a608318/aria/orchestrator/context/toolbelt.py
--
diff --git a/aria/orchestrator/context/toolbelt.py 
b/aria/orchestrator/context/toolbelt.py
index ceaeb72..ae0e1ff 100644
--- a/aria/orchestrator/context/toolbelt.py
+++ b/aria/orchestrator/context/toolbelt.py
@@ -33,7 +33,7 @@ class NodeToolBelt(object):
 :return:
 """
 assert isinstance(self._op_context, operation.NodeOperationContext)
-filters = {'target_node_instance_fk': 
self._op_context.node_instance.storage_id}
+filters = {'target_node_instance_storage_id': 
self._op_context.node_instance.storage_id}
 for relationship_instance in \
 
self._op_context.model.relationship_instance.iter(filters=filters):
 yield relationship_instance.source_node_instance

http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/blob/9a608318/aria/orchestrator/context/workflow.py
--
diff --git a/aria/orchestrator/context/workflow.py 
b/aria/orchestrator/context/workflow.py
index 0540590..5ec4044 100644
--- a/aria/orchestrator/context/workflow.py
+++ b/aria/orchestrator/context/workflow.py
@@ -53,8 +53,8 @@ class WorkflowContext(BaseContext):
 now = datetime.utcnow()
 execution = self.model.execution.model_cls(
 id=self._execution_id,
-blueprint_fk=self.blueprint.storage_id,
-deployment_fk=self.deployment.storage_id,
+blueprint_storage_id=self.blueprint.storage_id,
+deployment_storage_id=self.deployment.storage_id,
 workflow_id=self._workflow_id,
 created_at=now,
 status=execution_cls.PENDING,
@@ -69,7 +69,7 @@ class 

[GitHub] incubator-ariatosca pull request #31: ARIA-30-SQL-based-storage-implementati...

[mxmrlv]

Github user mxmrlv commented on a diff in the pull request:

https://github.com/apache/incubator-ariatosca/pull/31#discussion_r90779972
  
--- Diff: tests/orchestrator/workflows/core/test_engine.py ---
@@ -233,12 +212,13 @@ def mock_workflow(ctx, graph):
 
 class TestCancel(BaseTest):
 
+# TODO: what is up with this test?
 def test_cancel_started_execution(self, workflow_context, executor):
 number_of_tasks = 100
 
 @workflow
 def mock_workflow(ctx, graph):
-return graph.sequence(*(self._op(mock_sleep_task, ctx, 
inputs={'seconds': 0.1})
+return graph.sequence(*(self._op(mock_sleep_task, ctx, 
inputs={'seconds': 1})
--- End diff --

get back to


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-ariatosca pull request #31: ARIA-30-SQL-based-storage-implementati...

[mxmrlv]

Github user mxmrlv commented on a diff in the pull request:

https://github.com/apache/incubator-ariatosca/pull/31#discussion_r90777213
  
--- Diff: aria/storage/sql_mapi.py ---
@@ -0,0 +1,361 @@
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements.  See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License.  You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+"""
+SQLAlchemy based MAPI
+"""
+
+from sqlalchemy.exc import SQLAlchemyError
+from sqlalchemy.sql.elements import Label
+
+from aria.utils.collections import OrderedDict
+
+from aria.storage import (
+api,
+exceptions
+)
+
+
+DEFAULT_SQL_DIALECT = 'sqlite'
+
+
+class SQLAlchemyModelAPI(api.ModelAPI):
+"""
+SQL based MAPI.
+"""
+
+def __init__(self,
+ engine,
+ session,
+ **kwargs):
+super(SQLAlchemyModelAPI, self).__init__(**kwargs)
+self._engine = engine
+self._session = session
+
+def get(self, entry_id, include=None, filters=None, locking=False, 
**kwargs):
+"""Return a single result based on the model class and element ID
+"""
+filters = filters or {'id': entry_id}
+query = self._get_query(include, filters)
+if locking:
+query = query.with_for_update()
+result = query.first()
+
+if not result:
+raise exceptions.StorageError(
+'Requested {0} with ID `{1}` was not found'
+.format(self.model_cls.__name__, entry_id)
+)
+return result
+
+def iter(self,
+ include=None,
+ filters=None,
+ pagination=None,
+ sort=None,
+ **kwargs):
+"""Return a (possibly empty) list of `model_class` results
+"""
+query = self._get_query(include, filters, sort)
+
+results, _, _, _ = self._paginate(query, pagination)
+
+for result in results:
+yield result
+
+def put(self, entry, **kwargs):
+"""Create a `model_class` instance from a serializable `model` 
object
+
+:param entry: A dict with relevant kwargs, or an instance of a 
class
+that has a `to_dict` method, and whose attributes match the columns
+of `model_class` (might also my just an instance of `model_class`)
+:return: An instance of `model_class`
+"""
+self._session.add(entry)
+self._safe_commit()
+return entry
+
+def delete(self, entry_id, filters=None, **kwargs):
+"""Delete a single result based on the model class and element ID
+"""
+try:
+instance = self.get(
+entry_id,
+filters=filters
+)
+except exceptions.StorageError:
+raise exceptions.StorageError(
+'Could not delete {0} with ID `{1}` - element not found'
+.format(
+self.model_cls.__name__,
+entry_id
+)
+)
+self._load_properties(instance)
+self._session.delete(instance)
+self._safe_commit()
+return instance
+
+# TODO: this might need rework
+def update(self, entry, **kwargs):
+"""Add `instance` to the DB session, and attempt to commit
+
+:return: The updated instance
+"""
+return self.put(entry)
+
+def refresh(self, entry):
+"""Reload the instance with fresh information from the DB
+
+:param entry: Instance to be re-loaded from the DB
+:return: The refreshed instance
+"""
+self._session.refresh(entry)
+self._load_properties(entry)
+return entry
+
+def _destroy_connection(self):
+pass
+
+def _establish_connection(self):
+pass
+
+

[GitHub] incubator-ariatosca pull request #31: ARIA-30-SQL-based-storage-implementati...

[mxmrlv]

Github user mxmrlv commented on a diff in the pull request:

https://github.com/apache/incubator-ariatosca/pull/31#discussion_r90778725
  
--- Diff: aria/storage/models.py ---
@@ -422,23 +531,55 @@ def validate_max_attempts(_, value, *args):
 SUCCESS,
 FAILED,
 )
+
 WAIT_STATES = [PENDING, RETRYING]
 END_STATES = [SUCCESS, FAILED]
+
+@validates('max_attempts')
+def validate_max_attempts(self, _, value): 
 # pylint: disable=no-self-use
+"""Validates that max attempts is either -1 or a positive number"""
+if value < 1 and value != Task.INFINITE_RETRIES:
+raise ValueError('Max attempts can be either -1 (infinite) or 
any positive number. '
+ 'Got {value}'.format(value=value))
+return value
+
 INFINITE_RETRIES = -1
 
-id = Field(type=basestring, default=uuid_generator)
-status = Field(type=basestring, choices=STATES, default=PENDING)
-execution_id = Field(type=basestring)
-due_at = Field(type=datetime, default=datetime.utcnow)
-started_at = Field(type=datetime, default=None)
-ended_at = Field(type=datetime, default=None)
-max_attempts = Field(type=int, default=1, 
validation_func=_Validation.validate_max_attempts)
-retry_count = Field(type=int, default=0)
-retry_interval = Field(type=(int, float), default=0)
-ignore_failure = Field(type=bool, default=False)
+status = Column(Enum(*STATES), name='status', default=PENDING)
+
+execution_id = Column(String)
+due_at = Column(DateTime, default=datetime.utcnow, nullable=True)
+started_at = Column(DateTime, default=None, nullable=True)
+ended_at = Column(DateTime, default=None, nullable=True)
+max_attempts = Column(Integer, default=1)
+retry_count = Column(Integer, default=0)
+retry_interval = Column(Float, default=0)
+ignore_failure = Column(Boolean, default=False)
 
 # Operation specific fields
-name = Field(type=basestring)
-operation_mapping = Field(type=basestring)
-actor = Field()
-inputs = Field(type=dict, default=lambda: {})
+name = Column(String)
+operation_mapping = Column(String)
+inputs = Column(MutableDict.as_mutable(Dict))
+
+@property
+def actor_storage_id(self):
--- End diff --

remove


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-ariatosca pull request #31: ARIA-30-SQL-based-storage-implementati...

[mxmrlv]

Github user mxmrlv commented on a diff in the pull request:

https://github.com/apache/incubator-ariatosca/pull/31#discussion_r90778979
  
--- Diff: aria/storage/structures.py ---
@@ -27,281 +27,189 @@
 * Model - abstract model implementation.
 """
 import json
-from itertools import count
-from uuid import uuid4
-
-from .exceptions import StorageError
-from ..logger import LoggerMixin
-from ..utils.validation import ValidatorMixin
-
-__all__ = (
-'uuid_generator',
-'Field',
-'IterField',
-'PointerField',
-'IterPointerField',
-'Model',
-'Storage',
+
+from sqlalchemy import VARCHAR
+from sqlalchemy.ext.mutable import Mutable
+from sqlalchemy.orm import relationship, backref
+from sqlalchemy.ext.declarative import declarative_base
+# pylint: disable=unused-import
+from sqlalchemy.ext.associationproxy import association_proxy
+from sqlalchemy import (
+schema,
+Column,
+Integer,
+Text,
+DateTime,
+Boolean,
+Enum,
+String,
+PickleType,
+Float,
+TypeDecorator,
+ForeignKey,
+orm,
 )
 
+Model = declarative_base()
 
-def uuid_generator():
-"""
-wrapper function which generates ids
-"""
-return str(uuid4())
 
+def foreign_key(foreign_key_column, nullable=False):
+"""Return a ForeignKey object with the relevant
 
-class Field(ValidatorMixin):
+:param foreign_key_column: Unique id column in the parent table
+:param nullable: Should the column be allowed to remain empty
 """
-A single field implementation
+return Column(
+ForeignKey(foreign_key_column, ondelete='CASCADE'),
+nullable=nullable
+)
+
+
+def one_to_many_relationship(child_class,
+ parent_class,
+ foreign_key_column,
+ backreference=None):
+"""Return a one-to-many SQL relationship object
+Meant to be used from inside the *child* object
+
+:param parent_class: Class of the parent table
+:param child_class: Class of the child table
+:param foreign_key_column: The column of the foreign key
+:param backreference: The name to give to the reference to the child
 """
-NO_DEFAULT = 'NO_DEFAULT'
-
-try:
-# python 3 syntax
-_next_id = count().__next__
-except AttributeError:
-# python 2 syntax
-_next_id = count().next
-_ATTRIBUTE_NAME = '_cache_{0}'.format
-
-def __init__(
-self,
-type=None,
-choices=(),
-validation_func=None,
-default=NO_DEFAULT,
-**kwargs):
-"""
-Simple field manager.
-
-:param type: possible type of the field.
-:param choices: a set of possible field values.
-:param default: default field value.
-:param kwargs: kwargs to be passed to next in line classes.
-"""
-self.type = type
-self.choices = choices
-self.default = default
-self.validation_func = validation_func
-super(Field, self).__init__(**kwargs)
-
-def __get__(self, instance, owner):
-if instance is None:
-return self
-field_name = self._field_name(instance)
-try:
-return getattr(instance, self._ATTRIBUTE_NAME(field_name))
-except AttributeError as exc:
-if self.default == self.NO_DEFAULT:
-raise AttributeError(
-str(exc).replace(self._ATTRIBUTE_NAME(field_name), 
field_name))
-
-default_value = self.default() if callable(self.default) else 
self.default
-setattr(instance, self._ATTRIBUTE_NAME(field_name), default_value)
-return default_value
-
-def __set__(self, instance, value):
-field_name = self._field_name(instance)
-self.validate_value(field_name, value, instance)
-setattr(instance, self._ATTRIBUTE_NAME(field_name), value)
-
-def validate_value(self, name, value, instance):
-"""
-Validates the value of the field.
-
-:param name: the name of the field.
-:param value: the value of the field.
-:param instance: the instance containing the field.
-"""
-if self.default != self.NO_DEFAULT and value == self.default:
-return
-if self.type:
-self.validate_instance(name, value, self.type)
-if self.choices:
- 

[GitHub] incubator-ariatosca pull request #31: ARIA-30-SQL-based-storage-implementati...

[mxmrlv]

Github user mxmrlv commented on a diff in the pull request:

https://github.com/apache/incubator-ariatosca/pull/31#discussion_r90778697
  
--- Diff: aria/storage/models.py ---
@@ -148,265 +302,220 @@ def __lt__(self, other):
 return False
 
 
-class DeploymentUpdate(Model):
+class DeploymentModification(SQLModelBase):
 """
-A Model which represents a deployment update
+Deployment modification model representation.
 """
-INITIALIZING = 'initializing'
-SUCCESSFUL = 'successful'
-UPDATING = 'updating'
-FINALIZING = 'finalizing'
-EXECUTING_WORKFLOW = 'executing_workflow'
-FAILED = 'failed'
+__tablename__ = 'deployment_modifications'
 
-STATES = [
-INITIALIZING,
-SUCCESSFUL,
-UPDATING,
-FINALIZING,
-EXECUTING_WORKFLOW,
-FAILED,
-]
-
-# '{0}-{1}'.format(kwargs['deployment_id'], uuid4())
-id = Field(type=basestring, default=uuid_generator)
-deployment_id = Field(type=basestring)
-state = Field(type=basestring, choices=STATES, default=INITIALIZING)
-deployment_plan = Field()
-deployment_update_nodes = Field(default=None)
-deployment_update_node_instances = Field(default=None)
-deployment_update_deployment = Field(default=None)
-modified_entity_ids = Field(default=None)
-execution_id = Field(type=basestring)
-steps = IterPointerField(type=DeploymentUpdateStep, default=())
-
-
-class Execution(Model):
-"""
-A Model which represents an execution
-"""
+STARTED = 'started'
+FINISHED = 'finished'
+ROLLEDBACK = 'rolledback'
 
-class _Validation(object):
-
-@staticmethod
-def execution_status_transition_validation(_, value, instance):
-"""Validation function that verifies execution status 
transitions are OK"""
-try:
-current_status = instance.status
-except AttributeError:
-return
-valid_transitions = 
Execution.VALID_TRANSITIONS.get(current_status, [])
-if current_status != value and value not in valid_transitions:
-raise ValueError('Cannot change execution status from 
{current} to {new}'.format(
-current=current_status,
-new=value))
+STATES = [STARTED, FINISHED, ROLLEDBACK]
+END_STATES = [FINISHED, ROLLEDBACK]
 
-TERMINATED = 'terminated'
-FAILED = 'failed'
-CANCELLED = 'cancelled'
-PENDING = 'pending'
-STARTED = 'started'
-CANCELLING = 'cancelling'
-STATES = (
-TERMINATED,
-FAILED,
-CANCELLED,
-PENDING,
-STARTED,
-CANCELLING,
-)
-END_STATES = [TERMINATED, FAILED, CANCELLED]
-ACTIVE_STATES = [state for state in STATES if state not in END_STATES]
-VALID_TRANSITIONS = {
-PENDING: [STARTED, CANCELLED],
-STARTED: END_STATES + [CANCELLING],
-CANCELLING: END_STATES
-}
+deployment_fk = foreign_key(Deployment.storage_id)
+_private_fields = ['deployment_fk']
+deployment_id = association_proxy('deployment', 'id')
 
-id = Field(type=basestring, default=uuid_generator)
-status = Field(type=basestring, choices=STATES,
-   
validation_func=_Validation.execution_status_transition_validation)
-deployment_id = Field(type=basestring)
-workflow_id = Field(type=basestring)
-blueprint_id = Field(type=basestring)
-created_at = Field(type=datetime, default=datetime.utcnow)
-started_at = Field(type=datetime, default=None)
-ended_at = Field(type=datetime, default=None)
-error = Field(type=basestring, default=None)
-parameters = Field()
+context = Column(MutableDict.as_mutable(Dict))
+created_at = Column(DateTime, nullable=False, index=True)
+ended_at = Column(DateTime, index=True)
+modified_nodes = Column(MutableDict.as_mutable(Dict))
+node_instances = Column(MutableDict.as_mutable(Dict))
+status = Column(Enum(*STATES, name='deployment_modification_status'))
 
+@declared_attr
+def deployment(cls):
+return one_to_many_relationship(cls,
+Deployment,
+cls.deployment_fk,
+backreference='modifications')
 
-class Relationship(Model):
+
+class Node(SQLModelBase):
 """
-A Model which represents a relationship
+Node model representation.
 """
-id = 

[GitHub] incubator-ariatosca pull request #31: ARIA-30-SQL-based-storage-implementati...

[mxmrlv]

Github user mxmrlv commented on a diff in the pull request:

https://github.com/apache/incubator-ariatosca/pull/31#discussion_r90778632
  
--- Diff: aria/storage/models.py ---
@@ -148,265 +302,220 @@ def __lt__(self, other):
 return False
 
 
-class DeploymentUpdate(Model):
+class DeploymentModification(SQLModelBase):
 """
-A Model which represents a deployment update
+Deployment modification model representation.
 """
-INITIALIZING = 'initializing'
-SUCCESSFUL = 'successful'
-UPDATING = 'updating'
-FINALIZING = 'finalizing'
-EXECUTING_WORKFLOW = 'executing_workflow'
-FAILED = 'failed'
+__tablename__ = 'deployment_modifications'
 
-STATES = [
-INITIALIZING,
-SUCCESSFUL,
-UPDATING,
-FINALIZING,
-EXECUTING_WORKFLOW,
-FAILED,
-]
-
-# '{0}-{1}'.format(kwargs['deployment_id'], uuid4())
-id = Field(type=basestring, default=uuid_generator)
-deployment_id = Field(type=basestring)
-state = Field(type=basestring, choices=STATES, default=INITIALIZING)
-deployment_plan = Field()
-deployment_update_nodes = Field(default=None)
-deployment_update_node_instances = Field(default=None)
-deployment_update_deployment = Field(default=None)
-modified_entity_ids = Field(default=None)
-execution_id = Field(type=basestring)
-steps = IterPointerField(type=DeploymentUpdateStep, default=())
-
-
-class Execution(Model):
-"""
-A Model which represents an execution
-"""
+STARTED = 'started'
+FINISHED = 'finished'
+ROLLEDBACK = 'rolledback'
 
-class _Validation(object):
-
-@staticmethod
-def execution_status_transition_validation(_, value, instance):
-"""Validation function that verifies execution status 
transitions are OK"""
-try:
-current_status = instance.status
-except AttributeError:
-return
-valid_transitions = 
Execution.VALID_TRANSITIONS.get(current_status, [])
-if current_status != value and value not in valid_transitions:
-raise ValueError('Cannot change execution status from 
{current} to {new}'.format(
-current=current_status,
-new=value))
+STATES = [STARTED, FINISHED, ROLLEDBACK]
+END_STATES = [FINISHED, ROLLEDBACK]
 
-TERMINATED = 'terminated'
-FAILED = 'failed'
-CANCELLED = 'cancelled'
-PENDING = 'pending'
-STARTED = 'started'
-CANCELLING = 'cancelling'
-STATES = (
-TERMINATED,
-FAILED,
-CANCELLED,
-PENDING,
-STARTED,
-CANCELLING,
-)
-END_STATES = [TERMINATED, FAILED, CANCELLED]
-ACTIVE_STATES = [state for state in STATES if state not in END_STATES]
-VALID_TRANSITIONS = {
-PENDING: [STARTED, CANCELLED],
-STARTED: END_STATES + [CANCELLING],
-CANCELLING: END_STATES
-}
+deployment_fk = foreign_key(Deployment.storage_id)
+_private_fields = ['deployment_fk']
+deployment_id = association_proxy('deployment', 'id')
 
-id = Field(type=basestring, default=uuid_generator)
-status = Field(type=basestring, choices=STATES,
-   
validation_func=_Validation.execution_status_transition_validation)
-deployment_id = Field(type=basestring)
-workflow_id = Field(type=basestring)
-blueprint_id = Field(type=basestring)
-created_at = Field(type=datetime, default=datetime.utcnow)
-started_at = Field(type=datetime, default=None)
-ended_at = Field(type=datetime, default=None)
-error = Field(type=basestring, default=None)
-parameters = Field()
+context = Column(MutableDict.as_mutable(Dict))
+created_at = Column(DateTime, nullable=False, index=True)
+ended_at = Column(DateTime, index=True)
+modified_nodes = Column(MutableDict.as_mutable(Dict))
+node_instances = Column(MutableDict.as_mutable(Dict))
+status = Column(Enum(*STATES, name='deployment_modification_status'))
 
+@declared_attr
+def deployment(cls):
+return one_to_many_relationship(cls,
+Deployment,
+cls.deployment_fk,
+backreference='modifications')
 
-class Relationship(Model):
+
+class Node(SQLModelBase):
 """
-A Model which represents a relationship
+Node model representation.
 """
-id = 

[GitHub] incubator-ariatosca pull request #31: ARIA-30-SQL-based-storage-implementati...

[mxmrlv]

Github user mxmrlv commented on a diff in the pull request:

https://github.com/apache/incubator-ariatosca/pull/31#discussion_r90779218
  
--- Diff: aria/storage/structures.py ---
@@ -27,281 +27,189 @@
 * Model - abstract model implementation.
 """
 import json
-from itertools import count
-from uuid import uuid4
-
-from .exceptions import StorageError
-from ..logger import LoggerMixin
-from ..utils.validation import ValidatorMixin
-
-__all__ = (
-'uuid_generator',
-'Field',
-'IterField',
-'PointerField',
-'IterPointerField',
-'Model',
-'Storage',
+
+from sqlalchemy import VARCHAR
+from sqlalchemy.ext.mutable import Mutable
+from sqlalchemy.orm import relationship, backref
+from sqlalchemy.ext.declarative import declarative_base
+# pylint: disable=unused-import
+from sqlalchemy.ext.associationproxy import association_proxy
+from sqlalchemy import (
+schema,
+Column,
+Integer,
+Text,
+DateTime,
+Boolean,
+Enum,
+String,
+PickleType,
+Float,
+TypeDecorator,
+ForeignKey,
+orm,
 )
 
+Model = declarative_base()
 
-def uuid_generator():
-"""
-wrapper function which generates ids
-"""
-return str(uuid4())
 
+def foreign_key(foreign_key_column, nullable=False):
+"""Return a ForeignKey object with the relevant
 
-class Field(ValidatorMixin):
+:param foreign_key_column: Unique id column in the parent table
+:param nullable: Should the column be allowed to remain empty
 """
-A single field implementation
+return Column(
+ForeignKey(foreign_key_column, ondelete='CASCADE'),
+nullable=nullable
+)
+
+
+def one_to_many_relationship(child_class,
+ parent_class,
+ foreign_key_column,
+ backreference=None):
+"""Return a one-to-many SQL relationship object
+Meant to be used from inside the *child* object
+
+:param parent_class: Class of the parent table
+:param child_class: Class of the child table
+:param foreign_key_column: The column of the foreign key
+:param backreference: The name to give to the reference to the child
 """
-NO_DEFAULT = 'NO_DEFAULT'
-
-try:
-# python 3 syntax
-_next_id = count().__next__
-except AttributeError:
-# python 2 syntax
-_next_id = count().next
-_ATTRIBUTE_NAME = '_cache_{0}'.format
-
-def __init__(
-self,
-type=None,
-choices=(),
-validation_func=None,
-default=NO_DEFAULT,
-**kwargs):
-"""
-Simple field manager.
-
-:param type: possible type of the field.
-:param choices: a set of possible field values.
-:param default: default field value.
-:param kwargs: kwargs to be passed to next in line classes.
-"""
-self.type = type
-self.choices = choices
-self.default = default
-self.validation_func = validation_func
-super(Field, self).__init__(**kwargs)
-
-def __get__(self, instance, owner):
-if instance is None:
-return self
-field_name = self._field_name(instance)
-try:
-return getattr(instance, self._ATTRIBUTE_NAME(field_name))
-except AttributeError as exc:
-if self.default == self.NO_DEFAULT:
-raise AttributeError(
-str(exc).replace(self._ATTRIBUTE_NAME(field_name), 
field_name))
-
-default_value = self.default() if callable(self.default) else 
self.default
-setattr(instance, self._ATTRIBUTE_NAME(field_name), default_value)
-return default_value
-
-def __set__(self, instance, value):
-field_name = self._field_name(instance)
-self.validate_value(field_name, value, instance)
-setattr(instance, self._ATTRIBUTE_NAME(field_name), value)
-
-def validate_value(self, name, value, instance):
-"""
-Validates the value of the field.
-
-:param name: the name of the field.
-:param value: the value of the field.
-:param instance: the instance containing the field.
-"""
-if self.default != self.NO_DEFAULT and value == self.default:
-return
-if self.type:
-self.validate_instance(name, value, self.type)
-if self.choices:
- 

[GitHub] incubator-ariatosca pull request #31: ARIA-30-SQL-based-storage-implementati...

[mxmrlv]

Github user mxmrlv commented on a diff in the pull request:

https://github.com/apache/incubator-ariatosca/pull/31#discussion_r90779014
  
--- Diff: aria/storage/structures.py ---
@@ -27,281 +27,189 @@
 * Model - abstract model implementation.
 """
 import json
-from itertools import count
-from uuid import uuid4
-
-from .exceptions import StorageError
-from ..logger import LoggerMixin
-from ..utils.validation import ValidatorMixin
-
-__all__ = (
-'uuid_generator',
-'Field',
-'IterField',
-'PointerField',
-'IterPointerField',
-'Model',
-'Storage',
+
+from sqlalchemy import VARCHAR
+from sqlalchemy.ext.mutable import Mutable
+from sqlalchemy.orm import relationship, backref
+from sqlalchemy.ext.declarative import declarative_base
+# pylint: disable=unused-import
+from sqlalchemy.ext.associationproxy import association_proxy
+from sqlalchemy import (
+schema,
+Column,
+Integer,
+Text,
+DateTime,
+Boolean,
+Enum,
+String,
+PickleType,
+Float,
+TypeDecorator,
+ForeignKey,
+orm,
 )
 
+Model = declarative_base()
 
-def uuid_generator():
-"""
-wrapper function which generates ids
-"""
-return str(uuid4())
 
+def foreign_key(foreign_key_column, nullable=False):
+"""Return a ForeignKey object with the relevant
 
-class Field(ValidatorMixin):
+:param foreign_key_column: Unique id column in the parent table
+:param nullable: Should the column be allowed to remain empty
 """
-A single field implementation
+return Column(
+ForeignKey(foreign_key_column, ondelete='CASCADE'),
+nullable=nullable
+)
+
+
+def one_to_many_relationship(child_class,
+ parent_class,
+ foreign_key_column,
+ backreference=None):
+"""Return a one-to-many SQL relationship object
+Meant to be used from inside the *child* object
+
+:param parent_class: Class of the parent table
+:param child_class: Class of the child table
+:param foreign_key_column: The column of the foreign key
+:param backreference: The name to give to the reference to the child
 """
-NO_DEFAULT = 'NO_DEFAULT'
-
-try:
-# python 3 syntax
-_next_id = count().__next__
-except AttributeError:
-# python 2 syntax
-_next_id = count().next
-_ATTRIBUTE_NAME = '_cache_{0}'.format
-
-def __init__(
-self,
-type=None,
-choices=(),
-validation_func=None,
-default=NO_DEFAULT,
-**kwargs):
-"""
-Simple field manager.
-
-:param type: possible type of the field.
-:param choices: a set of possible field values.
-:param default: default field value.
-:param kwargs: kwargs to be passed to next in line classes.
-"""
-self.type = type
-self.choices = choices
-self.default = default
-self.validation_func = validation_func
-super(Field, self).__init__(**kwargs)
-
-def __get__(self, instance, owner):
-if instance is None:
-return self
-field_name = self._field_name(instance)
-try:
-return getattr(instance, self._ATTRIBUTE_NAME(field_name))
-except AttributeError as exc:
-if self.default == self.NO_DEFAULT:
-raise AttributeError(
-str(exc).replace(self._ATTRIBUTE_NAME(field_name), 
field_name))
-
-default_value = self.default() if callable(self.default) else 
self.default
-setattr(instance, self._ATTRIBUTE_NAME(field_name), default_value)
-return default_value
-
-def __set__(self, instance, value):
-field_name = self._field_name(instance)
-self.validate_value(field_name, value, instance)
-setattr(instance, self._ATTRIBUTE_NAME(field_name), value)
-
-def validate_value(self, name, value, instance):
-"""
-Validates the value of the field.
-
-:param name: the name of the field.
-:param value: the value of the field.
-:param instance: the instance containing the field.
-"""
-if self.default != self.NO_DEFAULT and value == self.default:
-return
-if self.type:
-self.validate_instance(name, value, self.type)
-if self.choices:
- 

[GitHub] incubator-ariatosca pull request #31: ARIA-30-SQL-based-storage-implementati...

[mxmrlv]

Github user mxmrlv commented on a diff in the pull request:

https://github.com/apache/incubator-ariatosca/pull/31#discussion_r90778672
  
--- Diff: aria/storage/models.py ---
@@ -148,265 +302,220 @@ def __lt__(self, other):
 return False
 
 
-class DeploymentUpdate(Model):
+class DeploymentModification(SQLModelBase):
 """
-A Model which represents a deployment update
+Deployment modification model representation.
 """
-INITIALIZING = 'initializing'
-SUCCESSFUL = 'successful'
-UPDATING = 'updating'
-FINALIZING = 'finalizing'
-EXECUTING_WORKFLOW = 'executing_workflow'
-FAILED = 'failed'
+__tablename__ = 'deployment_modifications'
 
-STATES = [
-INITIALIZING,
-SUCCESSFUL,
-UPDATING,
-FINALIZING,
-EXECUTING_WORKFLOW,
-FAILED,
-]
-
-# '{0}-{1}'.format(kwargs['deployment_id'], uuid4())
-id = Field(type=basestring, default=uuid_generator)
-deployment_id = Field(type=basestring)
-state = Field(type=basestring, choices=STATES, default=INITIALIZING)
-deployment_plan = Field()
-deployment_update_nodes = Field(default=None)
-deployment_update_node_instances = Field(default=None)
-deployment_update_deployment = Field(default=None)
-modified_entity_ids = Field(default=None)
-execution_id = Field(type=basestring)
-steps = IterPointerField(type=DeploymentUpdateStep, default=())
-
-
-class Execution(Model):
-"""
-A Model which represents an execution
-"""
+STARTED = 'started'
+FINISHED = 'finished'
+ROLLEDBACK = 'rolledback'
 
-class _Validation(object):
-
-@staticmethod
-def execution_status_transition_validation(_, value, instance):
-"""Validation function that verifies execution status 
transitions are OK"""
-try:
-current_status = instance.status
-except AttributeError:
-return
-valid_transitions = 
Execution.VALID_TRANSITIONS.get(current_status, [])
-if current_status != value and value not in valid_transitions:
-raise ValueError('Cannot change execution status from 
{current} to {new}'.format(
-current=current_status,
-new=value))
+STATES = [STARTED, FINISHED, ROLLEDBACK]
+END_STATES = [FINISHED, ROLLEDBACK]
 
-TERMINATED = 'terminated'
-FAILED = 'failed'
-CANCELLED = 'cancelled'
-PENDING = 'pending'
-STARTED = 'started'
-CANCELLING = 'cancelling'
-STATES = (
-TERMINATED,
-FAILED,
-CANCELLED,
-PENDING,
-STARTED,
-CANCELLING,
-)
-END_STATES = [TERMINATED, FAILED, CANCELLED]
-ACTIVE_STATES = [state for state in STATES if state not in END_STATES]
-VALID_TRANSITIONS = {
-PENDING: [STARTED, CANCELLED],
-STARTED: END_STATES + [CANCELLING],
-CANCELLING: END_STATES
-}
+deployment_fk = foreign_key(Deployment.storage_id)
+_private_fields = ['deployment_fk']
+deployment_id = association_proxy('deployment', 'id')
 
-id = Field(type=basestring, default=uuid_generator)
-status = Field(type=basestring, choices=STATES,
-   
validation_func=_Validation.execution_status_transition_validation)
-deployment_id = Field(type=basestring)
-workflow_id = Field(type=basestring)
-blueprint_id = Field(type=basestring)
-created_at = Field(type=datetime, default=datetime.utcnow)
-started_at = Field(type=datetime, default=None)
-ended_at = Field(type=datetime, default=None)
-error = Field(type=basestring, default=None)
-parameters = Field()
+context = Column(MutableDict.as_mutable(Dict))
+created_at = Column(DateTime, nullable=False, index=True)
+ended_at = Column(DateTime, index=True)
+modified_nodes = Column(MutableDict.as_mutable(Dict))
+node_instances = Column(MutableDict.as_mutable(Dict))
+status = Column(Enum(*STATES, name='deployment_modification_status'))
 
+@declared_attr
+def deployment(cls):
+return one_to_many_relationship(cls,
+Deployment,
+cls.deployment_fk,
+backreference='modifications')
 
-class Relationship(Model):
+
+class Node(SQLModelBase):
 """
-A Model which represents a relationship
+Node model representation.
 """
-id = 

[GitHub] incubator-ariatosca pull request #31: ARIA-30-SQL-based-storage-implementati...

[mxmrlv]

Github user mxmrlv commented on a diff in the pull request:

https://github.com/apache/incubator-ariatosca/pull/31#discussion_r90779099
  
--- Diff: aria/storage/structures.py ---
@@ -27,281 +27,189 @@
 * Model - abstract model implementation.
 """
 import json
-from itertools import count
-from uuid import uuid4
-
-from .exceptions import StorageError
-from ..logger import LoggerMixin
-from ..utils.validation import ValidatorMixin
-
-__all__ = (
-'uuid_generator',
-'Field',
-'IterField',
-'PointerField',
-'IterPointerField',
-'Model',
-'Storage',
+
+from sqlalchemy import VARCHAR
+from sqlalchemy.ext.mutable import Mutable
+from sqlalchemy.orm import relationship, backref
+from sqlalchemy.ext.declarative import declarative_base
+# pylint: disable=unused-import
+from sqlalchemy.ext.associationproxy import association_proxy
+from sqlalchemy import (
+schema,
+Column,
+Integer,
+Text,
+DateTime,
+Boolean,
+Enum,
+String,
+PickleType,
+Float,
+TypeDecorator,
+ForeignKey,
+orm,
 )
 
+Model = declarative_base()
 
-def uuid_generator():
-"""
-wrapper function which generates ids
-"""
-return str(uuid4())
 
+def foreign_key(foreign_key_column, nullable=False):
+"""Return a ForeignKey object with the relevant
 
-class Field(ValidatorMixin):
+:param foreign_key_column: Unique id column in the parent table
+:param nullable: Should the column be allowed to remain empty
 """
-A single field implementation
+return Column(
+ForeignKey(foreign_key_column, ondelete='CASCADE'),
+nullable=nullable
+)
+
+
+def one_to_many_relationship(child_class,
+ parent_class,
+ foreign_key_column,
+ backreference=None):
+"""Return a one-to-many SQL relationship object
+Meant to be used from inside the *child* object
+
+:param parent_class: Class of the parent table
+:param child_class: Class of the child table
+:param foreign_key_column: The column of the foreign key
+:param backreference: The name to give to the reference to the child
 """
-NO_DEFAULT = 'NO_DEFAULT'
-
-try:
-# python 3 syntax
-_next_id = count().__next__
-except AttributeError:
-# python 2 syntax
-_next_id = count().next
-_ATTRIBUTE_NAME = '_cache_{0}'.format
-
-def __init__(
-self,
-type=None,
-choices=(),
-validation_func=None,
-default=NO_DEFAULT,
-**kwargs):
-"""
-Simple field manager.
-
-:param type: possible type of the field.
-:param choices: a set of possible field values.
-:param default: default field value.
-:param kwargs: kwargs to be passed to next in line classes.
-"""
-self.type = type
-self.choices = choices
-self.default = default
-self.validation_func = validation_func
-super(Field, self).__init__(**kwargs)
-
-def __get__(self, instance, owner):
-if instance is None:
-return self
-field_name = self._field_name(instance)
-try:
-return getattr(instance, self._ATTRIBUTE_NAME(field_name))
-except AttributeError as exc:
-if self.default == self.NO_DEFAULT:
-raise AttributeError(
-str(exc).replace(self._ATTRIBUTE_NAME(field_name), 
field_name))
-
-default_value = self.default() if callable(self.default) else 
self.default
-setattr(instance, self._ATTRIBUTE_NAME(field_name), default_value)
-return default_value
-
-def __set__(self, instance, value):
-field_name = self._field_name(instance)
-self.validate_value(field_name, value, instance)
-setattr(instance, self._ATTRIBUTE_NAME(field_name), value)
-
-def validate_value(self, name, value, instance):
-"""
-Validates the value of the field.
-
-:param name: the name of the field.
-:param value: the value of the field.
-:param instance: the instance containing the field.
-"""
-if self.default != self.NO_DEFAULT and value == self.default:
-return
-if self.type:
-self.validate_instance(name, value, self.type)
-if self.choices:
- 

[GitHub] incubator-ariatosca pull request #31: ARIA-30-SQL-based-storage-implementati...

[mxmrlv]

Github user mxmrlv commented on a diff in the pull request:

https://github.com/apache/incubator-ariatosca/pull/31#discussion_r90778768
  
--- Diff: aria/storage/models.py ---
@@ -422,23 +531,55 @@ def validate_max_attempts(_, value, *args):
 SUCCESS,
 FAILED,
 )
+
 WAIT_STATES = [PENDING, RETRYING]
 END_STATES = [SUCCESS, FAILED]
+
+@validates('max_attempts')
+def validate_max_attempts(self, _, value): 
 # pylint: disable=no-self-use
+"""Validates that max attempts is either -1 or a positive number"""
+if value < 1 and value != Task.INFINITE_RETRIES:
+raise ValueError('Max attempts can be either -1 (infinite) or 
any positive number. '
+ 'Got {value}'.format(value=value))
+return value
+
 INFINITE_RETRIES = -1
 
-id = Field(type=basestring, default=uuid_generator)
-status = Field(type=basestring, choices=STATES, default=PENDING)
-execution_id = Field(type=basestring)
-due_at = Field(type=datetime, default=datetime.utcnow)
-started_at = Field(type=datetime, default=None)
-ended_at = Field(type=datetime, default=None)
-max_attempts = Field(type=int, default=1, 
validation_func=_Validation.validate_max_attempts)
-retry_count = Field(type=int, default=0)
-retry_interval = Field(type=(int, float), default=0)
-ignore_failure = Field(type=bool, default=False)
+status = Column(Enum(*STATES), name='status', default=PENDING)
+
+execution_id = Column(String)
+due_at = Column(DateTime, default=datetime.utcnow, nullable=True)
+started_at = Column(DateTime, default=None, nullable=True)
+ended_at = Column(DateTime, default=None, nullable=True)
+max_attempts = Column(Integer, default=1)
+retry_count = Column(Integer, default=0)
+retry_interval = Column(Float, default=0)
+ignore_failure = Column(Boolean, default=False)
 
 # Operation specific fields
-name = Field(type=basestring)
-operation_mapping = Field(type=basestring)
-actor = Field()
-inputs = Field(type=dict, default=lambda: {})
+name = Column(String)
+operation_mapping = Column(String)
+inputs = Column(MutableDict.as_mutable(Dict))
+
+@property
+def actor_storage_id(self):
+"""
+Return the actor storage id of the task
+:return:
+"""
+return self.node_instance_fk or self.relationship_instance_fk
+
+@property
+def actor(self):
+"""
+Return the actor of the task
+:return:
+"""
+return self.node_instance or self.relationship_instance
+
+def __init__(self, actor=None, **kwargs):
--- End diff --

2 class methods


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-ariatosca pull request #31: ARIA-30-SQL-based-storage-implementati...

[mxmrlv]

Github user mxmrlv commented on a diff in the pull request:

https://github.com/apache/incubator-ariatosca/pull/31#discussion_r90779283
  
--- Diff: aria/storage/structures.py ---
@@ -27,281 +27,189 @@
 * Model - abstract model implementation.
 """
 import json
-from itertools import count
-from uuid import uuid4
-
-from .exceptions import StorageError
-from ..logger import LoggerMixin
-from ..utils.validation import ValidatorMixin
-
-__all__ = (
-'uuid_generator',
-'Field',
-'IterField',
-'PointerField',
-'IterPointerField',
-'Model',
-'Storage',
+
+from sqlalchemy import VARCHAR
+from sqlalchemy.ext.mutable import Mutable
+from sqlalchemy.orm import relationship, backref
+from sqlalchemy.ext.declarative import declarative_base
+# pylint: disable=unused-import
+from sqlalchemy.ext.associationproxy import association_proxy
+from sqlalchemy import (
+schema,
+Column,
+Integer,
+Text,
+DateTime,
+Boolean,
+Enum,
+String,
+PickleType,
+Float,
+TypeDecorator,
+ForeignKey,
+orm,
 )
 
+Model = declarative_base()
 
-def uuid_generator():
-"""
-wrapper function which generates ids
-"""
-return str(uuid4())
 
+def foreign_key(foreign_key_column, nullable=False):
+"""Return a ForeignKey object with the relevant
 
-class Field(ValidatorMixin):
+:param foreign_key_column: Unique id column in the parent table
+:param nullable: Should the column be allowed to remain empty
 """
-A single field implementation
+return Column(
+ForeignKey(foreign_key_column, ondelete='CASCADE'),
+nullable=nullable
+)
+
+
+def one_to_many_relationship(child_class,
+ parent_class,
+ foreign_key_column,
+ backreference=None):
+"""Return a one-to-many SQL relationship object
+Meant to be used from inside the *child* object
+
+:param parent_class: Class of the parent table
+:param child_class: Class of the child table
+:param foreign_key_column: The column of the foreign key
+:param backreference: The name to give to the reference to the child
 """
-NO_DEFAULT = 'NO_DEFAULT'
-
-try:
-# python 3 syntax
-_next_id = count().__next__
-except AttributeError:
-# python 2 syntax
-_next_id = count().next
-_ATTRIBUTE_NAME = '_cache_{0}'.format
-
-def __init__(
-self,
-type=None,
-choices=(),
-validation_func=None,
-default=NO_DEFAULT,
-**kwargs):
-"""
-Simple field manager.
-
-:param type: possible type of the field.
-:param choices: a set of possible field values.
-:param default: default field value.
-:param kwargs: kwargs to be passed to next in line classes.
-"""
-self.type = type
-self.choices = choices
-self.default = default
-self.validation_func = validation_func
-super(Field, self).__init__(**kwargs)
-
-def __get__(self, instance, owner):
-if instance is None:
-return self
-field_name = self._field_name(instance)
-try:
-return getattr(instance, self._ATTRIBUTE_NAME(field_name))
-except AttributeError as exc:
-if self.default == self.NO_DEFAULT:
-raise AttributeError(
-str(exc).replace(self._ATTRIBUTE_NAME(field_name), 
field_name))
-
-default_value = self.default() if callable(self.default) else 
self.default
-setattr(instance, self._ATTRIBUTE_NAME(field_name), default_value)
-return default_value
-
-def __set__(self, instance, value):
-field_name = self._field_name(instance)
-self.validate_value(field_name, value, instance)
-setattr(instance, self._ATTRIBUTE_NAME(field_name), value)
-
-def validate_value(self, name, value, instance):
-"""
-Validates the value of the field.
-
-:param name: the name of the field.
-:param value: the value of the field.
-:param instance: the instance containing the field.
-"""
-if self.default != self.NO_DEFAULT and value == self.default:
-return
-if self.type:
-self.validate_instance(name, value, self.type)
-if self.choices:
- 

[GitHub] incubator-ariatosca pull request #31: ARIA-30-SQL-based-storage-implementati...

[mxmrlv]

Github user mxmrlv commented on a diff in the pull request:

https://github.com/apache/incubator-ariatosca/pull/31#discussion_r90779796
  
--- Diff: tests/mock/context.py ---
@@ -13,21 +13,59 @@
 # See the License for the specific language governing permissions and
 # limitations under the License.
 
+import pytest
+
+
 from aria import application_model_storage
 from aria.orchestrator import context
+from aria.storage.sql_mapi import SQLAlchemyModelAPI
+
+from tests.storage import get_sqlite_api_params
 
 from . import models
-from ..storage import InMemoryModelDriver
 
 
+@pytest.fixture
 def simple(**kwargs):
-storage = application_model_storage(InMemoryModelDriver())
-storage.setup()
-storage.blueprint.store(models.get_blueprint())
-storage.deployment.store(models.get_deployment())
+api_params = get_sqlite_api_params()
+model_storage = application_model_storage(SQLAlchemyModelAPI, 
api_params=api_params)
--- End diff --

api_kwargs


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-ariatosca pull request #31: ARIA-30-SQL-based-storage-implementati...

[mxmrlv]

Github user mxmrlv commented on a diff in the pull request:

https://github.com/apache/incubator-ariatosca/pull/31#discussion_r90779347
  
--- Diff: aria/storage/structures.py ---
@@ -27,281 +27,189 @@
 * Model - abstract model implementation.
 """
 import json
-from itertools import count
-from uuid import uuid4
-
-from .exceptions import StorageError
-from ..logger import LoggerMixin
-from ..utils.validation import ValidatorMixin
-
-__all__ = (
-'uuid_generator',
-'Field',
-'IterField',
-'PointerField',
-'IterPointerField',
-'Model',
-'Storage',
+
+from sqlalchemy import VARCHAR
+from sqlalchemy.ext.mutable import Mutable
+from sqlalchemy.orm import relationship, backref
+from sqlalchemy.ext.declarative import declarative_base
+# pylint: disable=unused-import
+from sqlalchemy.ext.associationproxy import association_proxy
+from sqlalchemy import (
+schema,
+Column,
+Integer,
+Text,
+DateTime,
+Boolean,
+Enum,
+String,
+PickleType,
+Float,
+TypeDecorator,
+ForeignKey,
+orm,
 )
 
+Model = declarative_base()
 
-def uuid_generator():
-"""
-wrapper function which generates ids
-"""
-return str(uuid4())
 
+def foreign_key(foreign_key_column, nullable=False):
+"""Return a ForeignKey object with the relevant
 
-class Field(ValidatorMixin):
+:param foreign_key_column: Unique id column in the parent table
+:param nullable: Should the column be allowed to remain empty
 """
-A single field implementation
+return Column(
+ForeignKey(foreign_key_column, ondelete='CASCADE'),
+nullable=nullable
+)
+
+
+def one_to_many_relationship(child_class,
+ parent_class,
+ foreign_key_column,
+ backreference=None):
+"""Return a one-to-many SQL relationship object
+Meant to be used from inside the *child* object
+
+:param parent_class: Class of the parent table
+:param child_class: Class of the child table
+:param foreign_key_column: The column of the foreign key
+:param backreference: The name to give to the reference to the child
 """
-NO_DEFAULT = 'NO_DEFAULT'
-
-try:
-# python 3 syntax
-_next_id = count().__next__
-except AttributeError:
-# python 2 syntax
-_next_id = count().next
-_ATTRIBUTE_NAME = '_cache_{0}'.format
-
-def __init__(
-self,
-type=None,
-choices=(),
-validation_func=None,
-default=NO_DEFAULT,
-**kwargs):
-"""
-Simple field manager.
-
-:param type: possible type of the field.
-:param choices: a set of possible field values.
-:param default: default field value.
-:param kwargs: kwargs to be passed to next in line classes.
-"""
-self.type = type
-self.choices = choices
-self.default = default
-self.validation_func = validation_func
-super(Field, self).__init__(**kwargs)
-
-def __get__(self, instance, owner):
-if instance is None:
-return self
-field_name = self._field_name(instance)
-try:
-return getattr(instance, self._ATTRIBUTE_NAME(field_name))
-except AttributeError as exc:
-if self.default == self.NO_DEFAULT:
-raise AttributeError(
-str(exc).replace(self._ATTRIBUTE_NAME(field_name), 
field_name))
-
-default_value = self.default() if callable(self.default) else 
self.default
-setattr(instance, self._ATTRIBUTE_NAME(field_name), default_value)
-return default_value
-
-def __set__(self, instance, value):
-field_name = self._field_name(instance)
-self.validate_value(field_name, value, instance)
-setattr(instance, self._ATTRIBUTE_NAME(field_name), value)
-
-def validate_value(self, name, value, instance):
-"""
-Validates the value of the field.
-
-:param name: the name of the field.
-:param value: the value of the field.
-:param instance: the instance containing the field.
-"""
-if self.default != self.NO_DEFAULT and value == self.default:
-return
-if self.type:
-self.validate_instance(name, value, self.type)
-if self.choices:
- 

[GitHub] incubator-ariatosca pull request #31: ARIA-30-SQL-based-storage-implementati...

[mxmrlv]

Github user mxmrlv commented on a diff in the pull request:

https://github.com/apache/incubator-ariatosca/pull/31#discussion_r90778551
  
--- Diff: aria/storage/models.py ---
@@ -148,265 +302,220 @@ def __lt__(self, other):
 return False
 
 
-class DeploymentUpdate(Model):
+class DeploymentModification(SQLModelBase):
 """
-A Model which represents a deployment update
+Deployment modification model representation.
 """
-INITIALIZING = 'initializing'
-SUCCESSFUL = 'successful'
-UPDATING = 'updating'
-FINALIZING = 'finalizing'
-EXECUTING_WORKFLOW = 'executing_workflow'
-FAILED = 'failed'
+__tablename__ = 'deployment_modifications'
 
-STATES = [
-INITIALIZING,
-SUCCESSFUL,
-UPDATING,
-FINALIZING,
-EXECUTING_WORKFLOW,
-FAILED,
-]
-
-# '{0}-{1}'.format(kwargs['deployment_id'], uuid4())
-id = Field(type=basestring, default=uuid_generator)
-deployment_id = Field(type=basestring)
-state = Field(type=basestring, choices=STATES, default=INITIALIZING)
-deployment_plan = Field()
-deployment_update_nodes = Field(default=None)
-deployment_update_node_instances = Field(default=None)
-deployment_update_deployment = Field(default=None)
-modified_entity_ids = Field(default=None)
-execution_id = Field(type=basestring)
-steps = IterPointerField(type=DeploymentUpdateStep, default=())
-
-
-class Execution(Model):
-"""
-A Model which represents an execution
-"""
+STARTED = 'started'
+FINISHED = 'finished'
+ROLLEDBACK = 'rolledback'
 
-class _Validation(object):
-
-@staticmethod
-def execution_status_transition_validation(_, value, instance):
-"""Validation function that verifies execution status 
transitions are OK"""
-try:
-current_status = instance.status
-except AttributeError:
-return
-valid_transitions = 
Execution.VALID_TRANSITIONS.get(current_status, [])
-if current_status != value and value not in valid_transitions:
-raise ValueError('Cannot change execution status from 
{current} to {new}'.format(
-current=current_status,
-new=value))
+STATES = [STARTED, FINISHED, ROLLEDBACK]
+END_STATES = [FINISHED, ROLLEDBACK]
 
-TERMINATED = 'terminated'
-FAILED = 'failed'
-CANCELLED = 'cancelled'
-PENDING = 'pending'
-STARTED = 'started'
-CANCELLING = 'cancelling'
-STATES = (
-TERMINATED,
-FAILED,
-CANCELLED,
-PENDING,
-STARTED,
-CANCELLING,
-)
-END_STATES = [TERMINATED, FAILED, CANCELLED]
-ACTIVE_STATES = [state for state in STATES if state not in END_STATES]
-VALID_TRANSITIONS = {
-PENDING: [STARTED, CANCELLED],
-STARTED: END_STATES + [CANCELLING],
-CANCELLING: END_STATES
-}
+deployment_fk = foreign_key(Deployment.storage_id)
+_private_fields = ['deployment_fk']
+deployment_id = association_proxy('deployment', 'id')
 
-id = Field(type=basestring, default=uuid_generator)
-status = Field(type=basestring, choices=STATES,
-   
validation_func=_Validation.execution_status_transition_validation)
-deployment_id = Field(type=basestring)
-workflow_id = Field(type=basestring)
-blueprint_id = Field(type=basestring)
-created_at = Field(type=datetime, default=datetime.utcnow)
-started_at = Field(type=datetime, default=None)
-ended_at = Field(type=datetime, default=None)
-error = Field(type=basestring, default=None)
-parameters = Field()
+context = Column(MutableDict.as_mutable(Dict))
+created_at = Column(DateTime, nullable=False, index=True)
+ended_at = Column(DateTime, index=True)
+modified_nodes = Column(MutableDict.as_mutable(Dict))
+node_instances = Column(MutableDict.as_mutable(Dict))
+status = Column(Enum(*STATES, name='deployment_modification_status'))
 
+@declared_attr
+def deployment(cls):
+return one_to_many_relationship(cls,
+Deployment,
+cls.deployment_fk,
+backreference='modifications')
 
-class Relationship(Model):
+
+class Node(SQLModelBase):
 """
-A Model which represents a relationship
+Node model representation.
 """
-id = 

[GitHub] incubator-ariatosca pull request #31: ARIA-30-SQL-based-storage-implementati...

[mxmrlv]

Github user mxmrlv commented on a diff in the pull request:

https://github.com/apache/incubator-ariatosca/pull/31#discussion_r90780128
  
--- Diff: tests/storage/test_models.py ---
@@ -187,111 +99,18 @@ def test_deployment_update_step_model():
 
 assert remove_rel < remove_node < add_node < add_rel
 assert not add_node < None
-# TODO fix logic here so that pylint is happy
-# assert not modify_node < modify_rel and not modify_rel < modify_node
-
-
-def _relationship(id=''):
-return Relationship(
-id='rel{0}'.format(id),
-target_id='target{0}'.format(id),
-source_id='source{0}'.format(id),
-source_interfaces={},
-source_operations={},
-target_interfaces={},
-target_operations={},
-type='type{0}'.format(id),
-type_hierarchy=[],
-properties={})
-
-
-def test_relationships():
-relationships = [_relationship(index) for index in xrange(3)]
-
-node = Node(
-blueprint_id='blueprint_id',
-type='type',
-type_hierarchy=None,
-number_of_instances=1,
-planned_number_of_instances=1,
-deploy_number_of_instances=1,
-properties={},
-operations={},
-relationships=relationships,
-min_number_of_instances=1,
-max_number_of_instances=1)
-
-for index in xrange(3):
-assert relationships[index] is \
-   
next(node.relationships_by_target('target{0}'.format(index)))
-
-relationship = _relationship()
-
-node = Node(
--- End diff --

test every model and every field in every model


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-ariatosca pull request #31: ARIA-30-SQL-based-storage-implementati...

[mxmrlv]

Github user mxmrlv commented on a diff in the pull request:

https://github.com/apache/incubator-ariatosca/pull/31#discussion_r90778657
  
--- Diff: aria/storage/models.py ---
@@ -148,265 +302,220 @@ def __lt__(self, other):
 return False
 
 
-class DeploymentUpdate(Model):
+class DeploymentModification(SQLModelBase):
 """
-A Model which represents a deployment update
+Deployment modification model representation.
 """
-INITIALIZING = 'initializing'
-SUCCESSFUL = 'successful'
-UPDATING = 'updating'
-FINALIZING = 'finalizing'
-EXECUTING_WORKFLOW = 'executing_workflow'
-FAILED = 'failed'
+__tablename__ = 'deployment_modifications'
 
-STATES = [
-INITIALIZING,
-SUCCESSFUL,
-UPDATING,
-FINALIZING,
-EXECUTING_WORKFLOW,
-FAILED,
-]
-
-# '{0}-{1}'.format(kwargs['deployment_id'], uuid4())
-id = Field(type=basestring, default=uuid_generator)
-deployment_id = Field(type=basestring)
-state = Field(type=basestring, choices=STATES, default=INITIALIZING)
-deployment_plan = Field()
-deployment_update_nodes = Field(default=None)
-deployment_update_node_instances = Field(default=None)
-deployment_update_deployment = Field(default=None)
-modified_entity_ids = Field(default=None)
-execution_id = Field(type=basestring)
-steps = IterPointerField(type=DeploymentUpdateStep, default=())
-
-
-class Execution(Model):
-"""
-A Model which represents an execution
-"""
+STARTED = 'started'
+FINISHED = 'finished'
+ROLLEDBACK = 'rolledback'
 
-class _Validation(object):
-
-@staticmethod
-def execution_status_transition_validation(_, value, instance):
-"""Validation function that verifies execution status 
transitions are OK"""
-try:
-current_status = instance.status
-except AttributeError:
-return
-valid_transitions = 
Execution.VALID_TRANSITIONS.get(current_status, [])
-if current_status != value and value not in valid_transitions:
-raise ValueError('Cannot change execution status from 
{current} to {new}'.format(
-current=current_status,
-new=value))
+STATES = [STARTED, FINISHED, ROLLEDBACK]
+END_STATES = [FINISHED, ROLLEDBACK]
 
-TERMINATED = 'terminated'
-FAILED = 'failed'
-CANCELLED = 'cancelled'
-PENDING = 'pending'
-STARTED = 'started'
-CANCELLING = 'cancelling'
-STATES = (
-TERMINATED,
-FAILED,
-CANCELLED,
-PENDING,
-STARTED,
-CANCELLING,
-)
-END_STATES = [TERMINATED, FAILED, CANCELLED]
-ACTIVE_STATES = [state for state in STATES if state not in END_STATES]
-VALID_TRANSITIONS = {
-PENDING: [STARTED, CANCELLED],
-STARTED: END_STATES + [CANCELLING],
-CANCELLING: END_STATES
-}
+deployment_fk = foreign_key(Deployment.storage_id)
+_private_fields = ['deployment_fk']
+deployment_id = association_proxy('deployment', 'id')
 
-id = Field(type=basestring, default=uuid_generator)
-status = Field(type=basestring, choices=STATES,
-   
validation_func=_Validation.execution_status_transition_validation)
-deployment_id = Field(type=basestring)
-workflow_id = Field(type=basestring)
-blueprint_id = Field(type=basestring)
-created_at = Field(type=datetime, default=datetime.utcnow)
-started_at = Field(type=datetime, default=None)
-ended_at = Field(type=datetime, default=None)
-error = Field(type=basestring, default=None)
-parameters = Field()
+context = Column(MutableDict.as_mutable(Dict))
+created_at = Column(DateTime, nullable=False, index=True)
+ended_at = Column(DateTime, index=True)
+modified_nodes = Column(MutableDict.as_mutable(Dict))
+node_instances = Column(MutableDict.as_mutable(Dict))
+status = Column(Enum(*STATES, name='deployment_modification_status'))
 
+@declared_attr
+def deployment(cls):
+return one_to_many_relationship(cls,
+Deployment,
+cls.deployment_fk,
+backreference='modifications')
 
-class Relationship(Model):
+
+class Node(SQLModelBase):
 """
-A Model which represents a relationship
+Node model representation.
 """
-id = 

[GitHub] incubator-ariatosca pull request #31: ARIA-30-SQL-based-storage-implementati...

[mxmrlv]

Github user mxmrlv commented on a diff in the pull request:

https://github.com/apache/incubator-ariatosca/pull/31#discussion_r90779244
  
--- Diff: aria/storage/structures.py ---
@@ -27,281 +27,189 @@
 * Model - abstract model implementation.
 """
 import json
-from itertools import count
-from uuid import uuid4
-
-from .exceptions import StorageError
-from ..logger import LoggerMixin
-from ..utils.validation import ValidatorMixin
-
-__all__ = (
-'uuid_generator',
-'Field',
-'IterField',
-'PointerField',
-'IterPointerField',
-'Model',
-'Storage',
+
+from sqlalchemy import VARCHAR
+from sqlalchemy.ext.mutable import Mutable
+from sqlalchemy.orm import relationship, backref
+from sqlalchemy.ext.declarative import declarative_base
+# pylint: disable=unused-import
+from sqlalchemy.ext.associationproxy import association_proxy
+from sqlalchemy import (
+schema,
+Column,
+Integer,
+Text,
+DateTime,
+Boolean,
+Enum,
+String,
+PickleType,
+Float,
+TypeDecorator,
+ForeignKey,
+orm,
 )
 
+Model = declarative_base()
 
-def uuid_generator():
-"""
-wrapper function which generates ids
-"""
-return str(uuid4())
 
+def foreign_key(foreign_key_column, nullable=False):
+"""Return a ForeignKey object with the relevant
 
-class Field(ValidatorMixin):
+:param foreign_key_column: Unique id column in the parent table
+:param nullable: Should the column be allowed to remain empty
 """
-A single field implementation
+return Column(
+ForeignKey(foreign_key_column, ondelete='CASCADE'),
+nullable=nullable
+)
+
+
+def one_to_many_relationship(child_class,
+ parent_class,
+ foreign_key_column,
+ backreference=None):
+"""Return a one-to-many SQL relationship object
+Meant to be used from inside the *child* object
+
+:param parent_class: Class of the parent table
+:param child_class: Class of the child table
+:param foreign_key_column: The column of the foreign key
+:param backreference: The name to give to the reference to the child
 """
-NO_DEFAULT = 'NO_DEFAULT'
-
-try:
-# python 3 syntax
-_next_id = count().__next__
-except AttributeError:
-# python 2 syntax
-_next_id = count().next
-_ATTRIBUTE_NAME = '_cache_{0}'.format
-
-def __init__(
-self,
-type=None,
-choices=(),
-validation_func=None,
-default=NO_DEFAULT,
-**kwargs):
-"""
-Simple field manager.
-
-:param type: possible type of the field.
-:param choices: a set of possible field values.
-:param default: default field value.
-:param kwargs: kwargs to be passed to next in line classes.
-"""
-self.type = type
-self.choices = choices
-self.default = default
-self.validation_func = validation_func
-super(Field, self).__init__(**kwargs)
-
-def __get__(self, instance, owner):
-if instance is None:
-return self
-field_name = self._field_name(instance)
-try:
-return getattr(instance, self._ATTRIBUTE_NAME(field_name))
-except AttributeError as exc:
-if self.default == self.NO_DEFAULT:
-raise AttributeError(
-str(exc).replace(self._ATTRIBUTE_NAME(field_name), 
field_name))
-
-default_value = self.default() if callable(self.default) else 
self.default
-setattr(instance, self._ATTRIBUTE_NAME(field_name), default_value)
-return default_value
-
-def __set__(self, instance, value):
-field_name = self._field_name(instance)
-self.validate_value(field_name, value, instance)
-setattr(instance, self._ATTRIBUTE_NAME(field_name), value)
-
-def validate_value(self, name, value, instance):
-"""
-Validates the value of the field.
-
-:param name: the name of the field.
-:param value: the value of the field.
-:param instance: the instance containing the field.
-"""
-if self.default != self.NO_DEFAULT and value == self.default:
-return
-if self.type:
-self.validate_instance(name, value, self.type)
-if self.choices:
- 

[GitHub] incubator-ariatosca pull request #31: ARIA-30-SQL-based-storage-implementati...

[mxmrlv]

Github user mxmrlv commented on a diff in the pull request:

https://github.com/apache/incubator-ariatosca/pull/31#discussion_r90778954
  
--- Diff: aria/storage/structures.py ---
@@ -27,281 +27,189 @@
 * Model - abstract model implementation.
 """
 import json
-from itertools import count
-from uuid import uuid4
-
-from .exceptions import StorageError
-from ..logger import LoggerMixin
-from ..utils.validation import ValidatorMixin
-
-__all__ = (
-'uuid_generator',
-'Field',
-'IterField',
-'PointerField',
-'IterPointerField',
-'Model',
-'Storage',
+
+from sqlalchemy import VARCHAR
+from sqlalchemy.ext.mutable import Mutable
+from sqlalchemy.orm import relationship, backref
+from sqlalchemy.ext.declarative import declarative_base
+# pylint: disable=unused-import
+from sqlalchemy.ext.associationproxy import association_proxy
+from sqlalchemy import (
+schema,
+Column,
+Integer,
+Text,
+DateTime,
+Boolean,
+Enum,
+String,
+PickleType,
+Float,
+TypeDecorator,
+ForeignKey,
+orm,
 )
 
+Model = declarative_base()
 
-def uuid_generator():
-"""
-wrapper function which generates ids
-"""
-return str(uuid4())
 
+def foreign_key(foreign_key_column, nullable=False):
+"""Return a ForeignKey object with the relevant
 
-class Field(ValidatorMixin):
+:param foreign_key_column: Unique id column in the parent table
+:param nullable: Should the column be allowed to remain empty
 """
-A single field implementation
+return Column(
+ForeignKey(foreign_key_column, ondelete='CASCADE'),
+nullable=nullable
+)
+
+
+def one_to_many_relationship(child_class,
+ parent_class,
+ foreign_key_column,
+ backreference=None):
+"""Return a one-to-many SQL relationship object
+Meant to be used from inside the *child* object
+
+:param parent_class: Class of the parent table
+:param child_class: Class of the child table
+:param foreign_key_column: The column of the foreign key
+:param backreference: The name to give to the reference to the child
 """
-NO_DEFAULT = 'NO_DEFAULT'
-
-try:
-# python 3 syntax
-_next_id = count().__next__
-except AttributeError:
-# python 2 syntax
-_next_id = count().next
-_ATTRIBUTE_NAME = '_cache_{0}'.format
-
-def __init__(
-self,
-type=None,
-choices=(),
-validation_func=None,
-default=NO_DEFAULT,
-**kwargs):
-"""
-Simple field manager.
-
-:param type: possible type of the field.
-:param choices: a set of possible field values.
-:param default: default field value.
-:param kwargs: kwargs to be passed to next in line classes.
-"""
-self.type = type
-self.choices = choices
-self.default = default
-self.validation_func = validation_func
-super(Field, self).__init__(**kwargs)
-
-def __get__(self, instance, owner):
-if instance is None:
-return self
-field_name = self._field_name(instance)
-try:
-return getattr(instance, self._ATTRIBUTE_NAME(field_name))
-except AttributeError as exc:
-if self.default == self.NO_DEFAULT:
-raise AttributeError(
-str(exc).replace(self._ATTRIBUTE_NAME(field_name), 
field_name))
-
-default_value = self.default() if callable(self.default) else 
self.default
-setattr(instance, self._ATTRIBUTE_NAME(field_name), default_value)
-return default_value
-
-def __set__(self, instance, value):
-field_name = self._field_name(instance)
-self.validate_value(field_name, value, instance)
-setattr(instance, self._ATTRIBUTE_NAME(field_name), value)
-
-def validate_value(self, name, value, instance):
-"""
-Validates the value of the field.
-
-:param name: the name of the field.
-:param value: the value of the field.
-:param instance: the instance containing the field.
-"""
-if self.default != self.NO_DEFAULT and value == self.default:
-return
-if self.type:
-self.validate_instance(name, value, self.type)
-if self.choices:
- 

[GitHub] incubator-ariatosca pull request #31: ARIA-30-SQL-based-storage-implementati...

[mxmrlv]

Github user mxmrlv commented on a diff in the pull request:

https://github.com/apache/incubator-ariatosca/pull/31#discussion_r90779747
  
--- Diff: aria/storage/models.py ---
@@ -59,66 +76,203 @@
 'Plugin',
 )
 
-# todo: sort this, maybe move from mgr or move from aria???
-ACTION_TYPES = ()
-ENTITY_TYPES = ()
+
+def uuid_generator():
--- End diff --

remove


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

Podling Report Reminder - December 2016

[johndament]

Dear podling,

This email was sent by an automated system on behalf of the Apache
Incubator PMC. It is an initial reminder to give you plenty of time to
prepare your quarterly board report.

The board meeting is scheduled for Wed, 21 December 2016, 10:30 am PDT.
The report for your podling will form a part of the Incubator PMC
report. The Incubator PMC requires your report to be submitted 2 weeks
before the board meeting, to allow sufficient time for review and
submission (Wed, December 07).

Please submit your report with sufficient time to allow the Incubator
PMC, and subsequently board members to review and digest. Again, the
very latest you should submit your report is 2 weeks prior to the board
meeting.

Thanks,

The Apache Incubator PMC

Submitting your Report

--

Your report should contain the following:

*   Your project name
*   A brief description of your project, which assumes no knowledge of
the project or necessarily of its field
*   A list of the three most important issues to address in the move
towards graduation.
*   Any issues that the Incubator PMC or ASF Board might wish/need to be
aware of
*   How has the community developed since the last report
*   How has the project developed since the last report.

This should be appended to the Incubator Wiki page at:

https://wiki.apache.org/incubator/December2016

Note: This is manually populated. You may need to wait a little before
this page is created from a template.

Mentors
---

Mentors should review reports for their project(s) and sign them off on
the Incubator wiki page. Signing off reports shows that you are
following the project - projects that are not signed may raise alarms
for the Incubator PMC.

Incubator PMC

[jira] [Created] (ARIATOSCA-38) Create contribution guide document

[Ran Ziv (JIRA)]

Ran Ziv created ARIATOSCA-38:


 Summary: Create contribution guide document
 Key: ARIATOSCA-38
 URL: https://issues.apache.org/jira/browse/ARIATOSCA-38
 Project: AriaTosca
  Issue Type: Task
Reporter: Ran Ziv
Assignee: Ran Ziv






--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (ARIATOSCA-31) Investigate consolidation of extensions mechanisms

[Ran Ziv (JIRA)]

 [ 
https://issues.apache.org/jira/browse/ARIATOSCA-31?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ran Ziv updated ARIATOSCA-31:
-
Assignee: Dan Kilman

> Investigate consolidation of extensions mechanisms
> --
>
> Key: ARIATOSCA-31
> URL: https://issues.apache.org/jira/browse/ARIATOSCA-31
> Project: AriaTosca
>  Issue Type: Task
>Reporter: Ran Ziv
>Assignee: Dan Kilman
>Priority: Critical
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

incubator-ariatosca git commit: updated the SQL MAPI

[mxmrlv]

Repository: incubator-ariatosca
Updated Branches:
  refs/heads/ARIA-30-SQL-based-storage-implementation 7b1c7d8dc -> 5ea0b6313


updated the SQL MAPI


Project: http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/repo
Commit: 
http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/commit/5ea0b631
Tree: http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/tree/5ea0b631
Diff: http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/diff/5ea0b631

Branch: refs/heads/ARIA-30-SQL-based-storage-implementation
Commit: 5ea0b63130b57bccf772286ee73d627bcc80841b
Parents: 7b1c7d8
Author: mxmrlv 
Authored: Sun Dec 4 15:37:41 2016 +0200
Committer: mxmrlv 
Committed: Sun Dec 4 15:38:17 2016 +0200

--
 aria/storage/sql_mapi.py | 176 ++
 1 file changed, 93 insertions(+), 83 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/blob/5ea0b631/aria/storage/sql_mapi.py
--
diff --git a/aria/storage/sql_mapi.py b/aria/storage/sql_mapi.py
index 1acfd80..25cc82f 100644
--- a/aria/storage/sql_mapi.py
+++ b/aria/storage/sql_mapi.py
@@ -17,19 +17,14 @@ SQLAlchemy based MAPI
 """
 
 from sqlalchemy.exc import SQLAlchemyError
-from sqlalchemy.sql.elements import Label
 
 from aria.utils.collections import OrderedDict
-
 from aria.storage import (
 api,
 exceptions
 )
 
 
-DEFAULT_SQL_DIALECT = 'sqlite'
-
-
 class SQLAlchemyModelAPI(api.ModelAPI):
 """
 SQL based MAPI.
@@ -102,12 +97,11 @@ class SQLAlchemyModelAPI(api.ModelAPI):
 entry_id
 )
 )
-self._load_properties(instance)
+self._load_relationships(instance)
 self._session.delete(instance)
 self._safe_commit()
 return instance
 
-# TODO: this might need rework
 def update(self, entry, **kwargs):
 """Add `instance` to the DB session, and attempt to commit
 
@@ -122,7 +116,7 @@ class SQLAlchemyModelAPI(api.ModelAPI):
 :return: The refreshed instance
 """
 self._session.refresh(entry)
-self._load_properties(entry)
+self._load_relationships(entry)
 return entry
 
 def _destroy_connection(self):
@@ -156,25 +150,68 @@ class SQLAlchemyModelAPI(api.ModelAPI):
 
 :param include: A (possibly empty) list of columns to include in
 the query
-:param joins: A (possibly empty) list of models on which the query
-should join
 :return: An SQLAlchemy AppenderQuery object
 """
-
 # If only some columns are included, query through the session object
 if include:
+# Make sure that attributes come before association proxies
+include.sort(key=lambda x: x.is_clause_element)
 query = self._session.query(*include)
 else:
 # If all columns should be returned, query directly from the model
 query = self._session.query(self.model_cls)
 
-# Add any joins that might be necessary
-for join_model in joins:
-query = query.join(join_model)
+if not self._skip_joining(joins, include):
+for join_table in joins:
+query = query.join(join_table)
 
 return query
 
 @staticmethod
+def _get_joins(model_class, columns):
+"""Get a list of all the tables on which we need to join
+
+:param columns: A set of all columns involved in the query
+"""
+joins = []  # Using a list instead of a set because order is important
+for column_name in columns:
+column = getattr(model_class, column_name)
+while not column.is_attribute:
+column = column.remote_attr
+if column.is_attribute:
+join_class = column.class_
+else:
+join_class = column.local_attr.class_
+
+# Don't add the same class more than once
+if join_class not in joins:
+joins.append(join_class)
+return joins
+
+@staticmethod
+def _skip_joining(joins, include):
+"""Dealing with an edge case where the only included column comes from
+an other table. In this case, we mustn't join on the same table again
+
+:param joins: A list of tables on which we're trying to join
+:param include: The list of
+:return: True if we need to skip joining
+"""
+if not joins:
+return True
+join_table_names = [t.__tablename__ for t in joins]
+
+if len(include) != 1:
+return False
+
+column = include[0]
+if column.is_clause_element:
+table_name = column.element.table.name
+ 

[GitHub] incubator-ariatosca pull request #31: ARIA-30-SQL-based-storage-implementati...

[ran-z]

Github user ran-z commented on a diff in the pull request:

https://github.com/apache/incubator-ariatosca/pull/31#discussion_r90773942
  
--- Diff: aria/storage/sql_mapi.py ---
@@ -0,0 +1,361 @@
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements.  See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License.  You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+"""
+SQLAlchemy based MAPI
+"""
+
+from sqlalchemy.exc import SQLAlchemyError
+from sqlalchemy.sql.elements import Label
+
+from aria.utils.collections import OrderedDict
+
+from aria.storage import (
+api,
+exceptions
+)
+
+
+DEFAULT_SQL_DIALECT = 'sqlite'
+
+
+class SQLAlchemyModelAPI(api.ModelAPI):
+"""
+SQL based MAPI.
+"""
+
+def __init__(self,
+ engine,
+ session,
+ **kwargs):
+super(SQLAlchemyModelAPI, self).__init__(**kwargs)
+self._engine = engine
+self._session = session
+
+def get(self, entry_id, include=None, filters=None, locking=False, 
**kwargs):
+"""Return a single result based on the model class and element ID
+"""
+filters = filters or {'id': entry_id}
--- End diff --

entry_id should always be used


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-ariatosca pull request #31: ARIA-30-SQL-based-storage-implementati...

[ran-z]

Github user ran-z commented on a diff in the pull request:

https://github.com/apache/incubator-ariatosca/pull/31#discussion_r90774109
  
--- Diff: aria/storage/sql_mapi.py ---
@@ -0,0 +1,361 @@
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements.  See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License.  You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+"""
+SQLAlchemy based MAPI
+"""
+
+from sqlalchemy.exc import SQLAlchemyError
+from sqlalchemy.sql.elements import Label
+
+from aria.utils.collections import OrderedDict
+
+from aria.storage import (
+api,
+exceptions
+)
+
+
+DEFAULT_SQL_DIALECT = 'sqlite'
+
+
+class SQLAlchemyModelAPI(api.ModelAPI):
+"""
+SQL based MAPI.
+"""
+
+def __init__(self,
+ engine,
+ session,
+ **kwargs):
+super(SQLAlchemyModelAPI, self).__init__(**kwargs)
+self._engine = engine
+self._session = session
+
+def get(self, entry_id, include=None, filters=None, locking=False, 
**kwargs):
--- End diff --

context manager for locking


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-ariatosca pull request #31: ARIA-30-SQL-based-storage-implementati...

[ran-z]

Github user ran-z commented on a diff in the pull request:

https://github.com/apache/incubator-ariatosca/pull/31#discussion_r90774280
  
--- Diff: aria/storage/sql_mapi.py ---
@@ -0,0 +1,361 @@
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements.  See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License.  You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+"""
+SQLAlchemy based MAPI
+"""
+
+from sqlalchemy.exc import SQLAlchemyError
+from sqlalchemy.sql.elements import Label
+
+from aria.utils.collections import OrderedDict
+
+from aria.storage import (
+api,
+exceptions
+)
+
+
+DEFAULT_SQL_DIALECT = 'sqlite'
+
+
+class SQLAlchemyModelAPI(api.ModelAPI):
+"""
+SQL based MAPI.
+"""
+
+def __init__(self,
+ engine,
+ session,
+ **kwargs):
+super(SQLAlchemyModelAPI, self).__init__(**kwargs)
+self._engine = engine
+self._session = session
+
+def get(self, entry_id, include=None, filters=None, locking=False, 
**kwargs):
+"""Return a single result based on the model class and element ID
+"""
+filters = filters or {'id': entry_id}
+query = self._get_query(include, filters)
+if locking:
+query = query.with_for_update()
+result = query.first()
+
+if not result:
+raise exceptions.StorageError(
+'Requested {0} with ID `{1}` was not found'
+.format(self.model_cls.__name__, entry_id)
+)
+return result
+
+def iter(self,
+ include=None,
+ filters=None,
+ pagination=None,
+ sort=None,
+ **kwargs):
+"""Return a (possibly empty) list of `model_class` results
+"""
+query = self._get_query(include, filters, sort)
+
+results, _, _, _ = self._paginate(query, pagination)
+
+for result in results:
+yield result
+
+def put(self, entry, **kwargs):
+"""Create a `model_class` instance from a serializable `model` 
object
+
+:param entry: A dict with relevant kwargs, or an instance of a 
class
+that has a `to_dict` method, and whose attributes match the columns
+of `model_class` (might also my just an instance of `model_class`)
+:return: An instance of `model_class`
+"""
+self._session.add(entry)
+self._safe_commit()
+return entry
+
+def delete(self, entry_id, filters=None, **kwargs):
--- End diff --

check usage for get/delete for filters
probably remove both


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-ariatosca pull request #31: ARIA-30-SQL-based-storage-implementati...

[ran-z]

Github user ran-z commented on a diff in the pull request:

https://github.com/apache/incubator-ariatosca/pull/31#discussion_r90774188
  
--- Diff: aria/storage/sql_mapi.py ---
@@ -0,0 +1,361 @@
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements.  See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License.  You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+"""
+SQLAlchemy based MAPI
+"""
+
+from sqlalchemy.exc import SQLAlchemyError
+from sqlalchemy.sql.elements import Label
+
+from aria.utils.collections import OrderedDict
+
+from aria.storage import (
+api,
+exceptions
+)
+
+
+DEFAULT_SQL_DIALECT = 'sqlite'
+
+
+class SQLAlchemyModelAPI(api.ModelAPI):
+"""
+SQL based MAPI.
+"""
+
+def __init__(self,
+ engine,
+ session,
+ **kwargs):
+super(SQLAlchemyModelAPI, self).__init__(**kwargs)
+self._engine = engine
+self._session = session
+
+def get(self, entry_id, include=None, filters=None, locking=False, 
**kwargs):
+"""Return a single result based on the model class and element ID
+"""
+filters = filters or {'id': entry_id}
+query = self._get_query(include, filters)
+if locking:
+query = query.with_for_update()
+result = query.first()
+
+if not result:
+raise exceptions.StorageError(
+'Requested {0} with ID `{1}` was not found'
+.format(self.model_cls.__name__, entry_id)
+)
+return result
+
+def iter(self,
--- End diff --

1) list
2) use paginate etc.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-ariatosca pull request #31: ARIA-30-SQL-based-storage-implementati...

[ran-z]

Github user ran-z commented on a diff in the pull request:

https://github.com/apache/incubator-ariatosca/pull/31#discussion_r90774557
  
--- Diff: aria/storage/sql_mapi.py ---
@@ -0,0 +1,361 @@
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements.  See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License.  You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+"""
+SQLAlchemy based MAPI
+"""
+
+from sqlalchemy.exc import SQLAlchemyError
+from sqlalchemy.sql.elements import Label
+
+from aria.utils.collections import OrderedDict
+
+from aria.storage import (
+api,
+exceptions
+)
+
+
+DEFAULT_SQL_DIALECT = 'sqlite'
+
+
+class SQLAlchemyModelAPI(api.ModelAPI):
+"""
+SQL based MAPI.
+"""
+
+def __init__(self,
+ engine,
+ session,
+ **kwargs):
+super(SQLAlchemyModelAPI, self).__init__(**kwargs)
+self._engine = engine
+self._session = session
+
+def get(self, entry_id, include=None, filters=None, locking=False, 
**kwargs):
+"""Return a single result based on the model class and element ID
+"""
+filters = filters or {'id': entry_id}
+query = self._get_query(include, filters)
+if locking:
+query = query.with_for_update()
+result = query.first()
+
+if not result:
+raise exceptions.StorageError(
+'Requested {0} with ID `{1}` was not found'
+.format(self.model_cls.__name__, entry_id)
+)
+return result
+
+def iter(self,
+ include=None,
+ filters=None,
+ pagination=None,
+ sort=None,
+ **kwargs):
+"""Return a (possibly empty) list of `model_class` results
+"""
+query = self._get_query(include, filters, sort)
+
+results, _, _, _ = self._paginate(query, pagination)
+
+for result in results:
+yield result
+
+def put(self, entry, **kwargs):
+"""Create a `model_class` instance from a serializable `model` 
object
+
+:param entry: A dict with relevant kwargs, or an instance of a 
class
+that has a `to_dict` method, and whose attributes match the columns
+of `model_class` (might also my just an instance of `model_class`)
+:return: An instance of `model_class`
+"""
+self._session.add(entry)
+self._safe_commit()
+return entry
+
+def delete(self, entry_id, filters=None, **kwargs):
+"""Delete a single result based on the model class and element ID
+"""
+try:
+instance = self.get(
+entry_id,
+filters=filters
+)
+except exceptions.StorageError:
+raise exceptions.StorageError(
+'Could not delete {0} with ID `{1}` - element not found'
+.format(
+self.model_cls.__name__,
+entry_id
+)
+)
+self._load_properties(instance)
+self._session.delete(instance)
+self._safe_commit()
+return instance
+
+# TODO: this might need rework
+def update(self, entry, **kwargs):
+"""Add `instance` to the DB session, and attempt to commit
+
+:return: The updated instance
+"""
+return self.put(entry)
+
+def refresh(self, entry):
+"""Reload the instance with fresh information from the DB
+
+:param entry: Instance to be re-loaded from the DB
+:return: The refreshed instance
+"""
+self._session.refresh(entry)
+self._load_properties(entry)
+return entry
+
+def _destroy_connection(self):
+pass
+
+def _establish_connection(self):
+pass
+
+

[GitHub] incubator-ariatosca pull request #31: ARIA-30-SQL-based-storage-implementati...

[ran-z]

Github user ran-z commented on a diff in the pull request:

https://github.com/apache/incubator-ariatosca/pull/31#discussion_r90773988
  
--- Diff: aria/storage/sql_mapi.py ---
@@ -0,0 +1,361 @@
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements.  See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License.  You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+"""
+SQLAlchemy based MAPI
+"""
+
+from sqlalchemy.exc import SQLAlchemyError
+from sqlalchemy.sql.elements import Label
+
+from aria.utils.collections import OrderedDict
+
+from aria.storage import (
+api,
+exceptions
+)
+
+
+DEFAULT_SQL_DIALECT = 'sqlite'
+
+
+class SQLAlchemyModelAPI(api.ModelAPI):
+"""
+SQL based MAPI.
+"""
+
+def __init__(self,
+ engine,
+ session,
+ **kwargs):
+super(SQLAlchemyModelAPI, self).__init__(**kwargs)
+self._engine = engine
+self._session = session
+
+def get(self, entry_id, include=None, filters=None, locking=False, 
**kwargs):
+"""Return a single result based on the model class and element ID
+"""
+filters = filters or {'id': entry_id}
+query = self._get_query(include, filters)
+if locking:
+query = query.with_for_update()
+result = query.first()
+
+if not result:
+raise exceptions.StorageError(
+'Requested {0} with ID `{1}` was not found'
--- End diff --

add filters


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-ariatosca pull request #31: ARIA-30-SQL-based-storage-implementati...

[ran-z]

Github user ran-z commented on a diff in the pull request:

https://github.com/apache/incubator-ariatosca/pull/31#discussion_r90774547
  
--- Diff: aria/storage/sql_mapi.py ---
@@ -0,0 +1,361 @@
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements.  See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License.  You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+"""
+SQLAlchemy based MAPI
+"""
+
+from sqlalchemy.exc import SQLAlchemyError
+from sqlalchemy.sql.elements import Label
+
+from aria.utils.collections import OrderedDict
+
+from aria.storage import (
+api,
+exceptions
+)
+
+
+DEFAULT_SQL_DIALECT = 'sqlite'
+
+
+class SQLAlchemyModelAPI(api.ModelAPI):
+"""
+SQL based MAPI.
+"""
+
+def __init__(self,
+ engine,
+ session,
+ **kwargs):
+super(SQLAlchemyModelAPI, self).__init__(**kwargs)
+self._engine = engine
+self._session = session
+
+def get(self, entry_id, include=None, filters=None, locking=False, 
**kwargs):
+"""Return a single result based on the model class and element ID
+"""
+filters = filters or {'id': entry_id}
+query = self._get_query(include, filters)
+if locking:
+query = query.with_for_update()
+result = query.first()
+
+if not result:
+raise exceptions.StorageError(
+'Requested {0} with ID `{1}` was not found'
+.format(self.model_cls.__name__, entry_id)
+)
+return result
+
+def iter(self,
+ include=None,
+ filters=None,
+ pagination=None,
+ sort=None,
+ **kwargs):
+"""Return a (possibly empty) list of `model_class` results
+"""
+query = self._get_query(include, filters, sort)
+
+results, _, _, _ = self._paginate(query, pagination)
+
+for result in results:
+yield result
+
+def put(self, entry, **kwargs):
+"""Create a `model_class` instance from a serializable `model` 
object
+
+:param entry: A dict with relevant kwargs, or an instance of a 
class
+that has a `to_dict` method, and whose attributes match the columns
+of `model_class` (might also my just an instance of `model_class`)
+:return: An instance of `model_class`
+"""
+self._session.add(entry)
+self._safe_commit()
+return entry
+
+def delete(self, entry_id, filters=None, **kwargs):
+"""Delete a single result based on the model class and element ID
+"""
+try:
+instance = self.get(
+entry_id,
+filters=filters
+)
+except exceptions.StorageError:
+raise exceptions.StorageError(
+'Could not delete {0} with ID `{1}` - element not found'
+.format(
+self.model_cls.__name__,
+entry_id
+)
+)
+self._load_properties(instance)
+self._session.delete(instance)
+self._safe_commit()
+return instance
+
+# TODO: this might need rework
+def update(self, entry, **kwargs):
+"""Add `instance` to the DB session, and attempt to commit
+
+:return: The updated instance
+"""
+return self.put(entry)
+
+def refresh(self, entry):
+"""Reload the instance with fresh information from the DB
+
+:param entry: Instance to be re-loaded from the DB
+:return: The refreshed instance
+"""
+self._session.refresh(entry)
+self._load_properties(entry)
+return entry
+
+def _destroy_connection(self):
+pass
+
+def _establish_connection(self):
+pass
+
+