Re: [openstack-dev] [Congress] hands on lab

[Alex Yip]

Hi all,

I updated the hands-on lab.


Here's the newest version of the instructions:

https://goo.gl/W0Rhcv


Here's the newest version of the VirtualBox image:

https://goo.gl/o062Kc​


Please give it a try and let me know if you run into any problems.

- Alex




From: Tim Hinrichs 
Sent: Monday, September 28, 2015 10:54 AM
To: OpenStack Development Mailing List (not for usage questions)
Subject: Re: [openstack-dev] [Congress] hands on lab

Hi Alex,

I went through the HOL.  Some feedback...

1.  When starting up, it paused for a while showing the following messages.  
Maybe that caused the issues I mention below.
Waiting for network configuration...
Waiting up to 60 more seconds for network configuration...

2.  Maybe spell out the first paragraph of "Starting the Devstack VM".  Maybe 
something like

To run the devstack VM, first install VirtualBox.  Then download the image from 
XXX and start the VM.  (From VirtualBox's File menu, Import Appliance and then 
choose the file you downloaded.)

Now you'll want to ssh to the VM from your laptop so it's easier to copy and 
paste.  (The VM is setup to use bridge networking with DHCP.) To find the IP 
address, login to the VM through the console.

username: congress
password: congress

Run ‘ifconfig’ to get the IP address of the VM for eth0

$ ifconfig eth0

Now open a terminal from your laptop and ssh to that IP address using the same 
username and password from above.



Now that devstack is running, you can point your laptop's browser to the VM's 
IP address you found earlier to use Horizon.

 3. Connection problems.
Horizon.  The browser brought up the login screen but gave me the message 
"Unable to establish connection to keystone endpoint." after I provided 
admin/password.

Congress.  From the congress log I see connection errors happening for keystone 
and neutron.

Looking at the Keystone screen log I don't see much of anything.  Here's the 
API log in its entirety.  Seems it hasn't gotten any requests today (Sept 28).


10.1.184.61 - - [17/Sep/2015:12:05:08 -0700] "POST /v2.0/tokens HTTP/1.1" 200 
4287 "-" "python-keystoneclient" 52799(us)

10.1.184.61 - - [17/Sep/2015:12:05:08 -0700] "GET /v3/auth/tokens HTTP/1.1" 200 
7655 "-" "python-keystoneclient" 65278(us)

10.1.184.61 - - [17/Sep/2015:12:05:08 -0700] "POST /v2.0/tokens HTTP/1.1" 200 
4287 "-" "python-keystoneclient" 63428(us)

10.1.184.61 - - [17/Sep/2015:12:05:08 -0700] "GET /v3/auth/tokens HTTP/1.1" 200 
7655 "-" "python-keystoneclient" 135994(us)

10.1.184.61 - - [17/Sep/2015:12:05:09 -0700] "POST /v2.0/tokens HTTP/1.1" 200 
4287 "-" "python-keystoneclient" 56581(us)

10.1.184.61 - - [17/Sep/2015:12:05:09 -0700] "GET /v3/auth/tokens HTTP/1.1" 200 
7655 "-" "python-keystoneclient" 55412(us)

10.1.184.61 - - [17/Sep/2015:12:05:12 -0700] "GET /v2.0/users HTTP/1.1" 200 
1679 "-" "python-keystoneclient" 13630(us)

10.1.184.61 - - [17/Sep/2015:12:05:12 -0700] "GET /v2.0/OS-KSADM/roles 
HTTP/1.1" 200 397 "-" "python-keystoneclient" 10940(us)

10.1.184.61 - - [17/Sep/2015:12:05:12 -0700] "GET /v2.0/tenants HTTP/1.1" 200 
752 "-" "python-keystoneclient" 12387(us)

The VM has eth0 bridged.  I can ping 
google.com<https://urldefense.proofpoint.com/v2/url?u=http-3A__google.com&d=BQMFaQ&c=Sqcl0Ez6M0X8aeM67LKIiDJAXVeAw-YihVMNtXt-uEs&r=djA1lFdIf0--GIJ_8gr44Q&m=ljhoO2bNd5MUext7mNU0kFU56cbIs54xfSPMRX5C8As&s=Vs9VhbXru7JNzccLA0wFNrG1XIDBDtDWe5ia39s12PQ&e=>
 from inside the VM; I can ssh to the VM from my laptop.

Any ideas what's going on?  (I'm trying to unstack/clean/stack to see if that 
works, but it'll take a while.)

Tim




On Thu, Sep 17, 2015 at 6:05 PM Alex Yip 
mailto:a...@vmware.com>> wrote:
Hi all,
I have created a VirtualBox VM that matches the Vancouver handson-lab here:

https://drive.google.com/file/d/0B94E7u1TIA8oTEdOQlFERkFwMUE/view?usp=sharing<https://urldefense.proofpoint.com/v2/url?u=https-3A__drive.google.com_file_d_0B94E7u1TIA8oTEdOQlFERkFwMUE_view-3Fusp-3Dsharing&d=BQMFaQ&c=Sqcl0Ez6M0X8aeM67LKIiDJAXVeAw-YihVMNtXt-uEs&r=djA1lFdIf0--GIJ_8gr44Q&m=ljhoO2bNd5MUext7mNU0kFU56cbIs54xfSPMRX5C8As&s=BduiBDRbMbjcRSkKCB6T0j4bG6w4QAoVxzIhWRKOXKk&e=>

There's also an updated instruction document here:

https://docs.google.com/document/d/1ispwf56bX8sy9T0KZyosdHrSR9WHEVA1oGEIYA22Orw/pub<https://urldefense.proofpoint.com/v2/url?u=https-3A__docs.google.com_document_d_1ispwf56bX8sy9T0KZyosdHrSR9WHEVA1oGEIYA22Orw_pub&d=BQMFaQ&c=Sqcl0Ez6M0X8aeM67LKIiDJAXVeAw-YihVMNtXt-uEs&r=djA1lFdIf0--GIJ_8gr44Q&m=ljhoO2bNd5MUext7mNU0kFU56cb

Re: [openstack-dev] [Congress] Congress Usecases VM

[Alex Yip]

I was able to make devstack run without a network connection by disabling 
tempest.  So, I think it uses the loopback IP address, and that does not 
change, so rejoin-stack.sh works without a network at all.


- Alex




From: Zhou, Zhenzan 
Sent: Thursday, September 24, 2015 4:56 PM
To: OpenStack Development Mailing List (not for usage questions)
Subject: Re: [openstack-dev] [Congress] Congress Usecases VM

Rejoin-stack.sh works only if its IP was not changed. So using NAT network and 
fixed ip inside the VM can help.

BR
Zhou Zhenzan

From: Alex Yip [mailto:a...@vmware.com]
Sent: Friday, September 25, 2015 01:37
To: OpenStack Development Mailing List (not for usage questions)
Subject: Re: [openstack-dev] [Congress] Congress Usecases VM


I have been using images, rather than snapshots.



It doesn't take that long to start up.  First, I boot the VM which takes a 
minute or so.  Then I run rejoin-stack.sh which takes just another minute or 
so.  It's really not that bad, and rejoin-stack.sh restores vms and openstack 
state that was running before.



- Alex






From: Shiv Haris mailto:sha...@brocade.com>>
Sent: Thursday, September 24, 2015 10:29 AM
To: OpenStack Development Mailing List (not for usage questions)
Subject: Re: [openstack-dev] [Congress] Congress Usecases VM

Hi Congress folks,

I am looking for ideas. We want the Openstack to be running when the user 
instantiates the Usecase-VM. However creating a OVA file is possible only when 
the VM is halted which means Openstack is not running and the user will have to 
run devstack again (which is time consuming) when the VM is restarted.

The option is to take a snapshot. It appears that taking a snapshot of the VM 
and using it in another setup is not very straight forward. It involves 
modifying the .vbox file and seems that it is prone to user errors. I am 
leaning towards halting the machine and generating an OVA file.

I am looking for suggestions ….

Thanks,

-Shiv


From: Shiv Haris [mailto:sha...@brocade.com]
Sent: Thursday, September 24, 2015 9:53 AM
To: OpenStack Development Mailing List (not for usage questions)
Subject: Re: [openstack-dev] [Congress] Congress Usecases VM

First of all I apologize for not making it at the meeting yesterday, could not 
cut short another overlapping meeting.

Also, Tim thanks for the feedback. I have addressed some of the issues you 
posed however I am still working on some of the subtle issues raised. Once I 
have addressed all I will post another VM by end of the week.

-Shiv


From: Tim Hinrichs [mailto:t...@styra.com]
Sent: Friday, September 18, 2015 5:14 PM
To: OpenStack Development Mailing List (not for usage questions)
Subject: Re: [openstack-dev] [Congress] Congress Usecases VM

It's great to have this available!  I think it'll help people understand what's 
going on MUCH more quickly.

Some thoughts.
- The image is 3GB, which took me 30 minutes to download.  Are all VMs this 
big?  I think we should finish this as a VM but then look into doing it with 
containers to make it EVEN easier for people to get started.

[shivharis] Yes, unfortunately that is the case. The disk size I set is 20GB – 
but the OVA compress the image and disk to 3 GB. I will looking at other 
options.


- It gave me an error about a missing shared directory when I started up.
[shivharis] will fix this

- I expected devstack to be running when I launched the VM.  devstack startup 
time is substantial, and if there's a problem, it's good to assume the user 
won't know how to fix it.  Is it possible to have devstack up and running when 
we start the VM?  That said, it started up fine for me.
[shivharis] OVA files can be created only when the VM is halted, so devstack 
will be down when you bring up  the VM. I agree a snapshot will be a better 
choice.

- It'd be good to have a README to explain how to use the use-case structure. 
It wasn't obvious to me.
[shivharis] added.

- The top-level dir of the Congress_Usecases folder has a Congress_Usecases 
folder within it.  I assume the inner one shouldn't be there?
[shivharis] my automation issues, fixed.

- When I ran the 10_install_policy.sh, it gave me a bunch of authorization 
problems.
[shivharis] fixed

But otherwise I think the setup looks reasonable.  Will there be an undo script 
so that we can run the use cases one after another without worrying about 
interactions?
[shivharis] tricky, will find some way out.

Tim

[shivharis] Thanks

On Fri, Sep 18, 2015 at 11:03 AM Shiv Haris 
mailto:sha...@brocade.com>> wrote:
Hi Congress folks,

BTW the login/password for the VM is vagrant/vagrant

-Shiv


From: Shiv Haris [mailto:sha...@brocade.com<mailto:sha...@brocade.com>]
Sent: Thursday, September 17, 2015 5:03 PM
To: openstack-dev@lists.openstack.org<mailto:openstack-dev@lists.openstack.org>
Subject: [openstack-dev] [Congress] Congres

Re: [openstack-dev] [Congress] Congress Usecases VM

[Alex Yip]

I have been using images, rather than snapshots.


It doesn't take that long to start up.  First, I boot the VM which takes a 
minute or so.  Then I run rejoin-stack.sh which takes just another minute or 
so.  It's really not that bad, and rejoin-stack.sh restores vms and openstack 
state that was running before.


- Alex




From: Shiv Haris 
Sent: Thursday, September 24, 2015 10:29 AM
To: OpenStack Development Mailing List (not for usage questions)
Subject: Re: [openstack-dev] [Congress] Congress Usecases VM

Hi Congress folks,

I am looking for ideas. We want the Openstack to be running when the user 
instantiates the Usecase-VM. However creating a OVA file is possible only when 
the VM is halted which means Openstack is not running and the user will have to 
run devstack again (which is time consuming) when the VM is restarted.

The option is to take a snapshot. It appears that taking a snapshot of the VM 
and using it in another setup is not very straight forward. It involves 
modifying the .vbox file and seems that it is prone to user errors. I am 
leaning towards halting the machine and generating an OVA file.

I am looking for suggestions ….

Thanks,

-Shiv


From: Shiv Haris [mailto:sha...@brocade.com]
Sent: Thursday, September 24, 2015 9:53 AM
To: OpenStack Development Mailing List (not for usage questions)
Subject: Re: [openstack-dev] [Congress] Congress Usecases VM

First of all I apologize for not making it at the meeting yesterday, could not 
cut short another overlapping meeting.

Also, Tim thanks for the feedback. I have addressed some of the issues you 
posed however I am still working on some of the subtle issues raised. Once I 
have addressed all I will post another VM by end of the week.

-Shiv


From: Tim Hinrichs [mailto:t...@styra.com]
Sent: Friday, September 18, 2015 5:14 PM
To: OpenStack Development Mailing List (not for usage questions)
Subject: Re: [openstack-dev] [Congress] Congress Usecases VM

It's great to have this available!  I think it'll help people understand what's 
going on MUCH more quickly.

Some thoughts.
- The image is 3GB, which took me 30 minutes to download.  Are all VMs this 
big?  I think we should finish this as a VM but then look into doing it with 
containers to make it EVEN easier for people to get started.

[shivharis] Yes, unfortunately that is the case. The disk size I set is 20GB – 
but the OVA compress the image and disk to 3 GB. I will looking at other 
options.


- It gave me an error about a missing shared directory when I started up.
[shivharis] will fix this

- I expected devstack to be running when I launched the VM.  devstack startup 
time is substantial, and if there's a problem, it's good to assume the user 
won't know how to fix it.  Is it possible to have devstack up and running when 
we start the VM?  That said, it started up fine for me.
[shivharis] OVA files can be created only when the VM is halted, so devstack 
will be down when you bring up  the VM. I agree a snapshot will be a better 
choice.

- It'd be good to have a README to explain how to use the use-case structure. 
It wasn't obvious to me.
[shivharis] added.

- The top-level dir of the Congress_Usecases folder has a Congress_Usecases 
folder within it.  I assume the inner one shouldn't be there?
[shivharis] my automation issues, fixed.

- When I ran the 10_install_policy.sh, it gave me a bunch of authorization 
problems.
[shivharis] fixed

But otherwise I think the setup looks reasonable.  Will there be an undo script 
so that we can run the use cases one after another without worrying about 
interactions?
[shivharis] tricky, will find some way out.

Tim

[shivharis] Thanks

On Fri, Sep 18, 2015 at 11:03 AM Shiv Haris 
mailto:sha...@brocade.com>> wrote:
Hi Congress folks,

BTW the login/password for the VM is vagrant/vagrant

-Shiv


From: Shiv Haris [mailto:sha...@brocade.com]
Sent: Thursday, September 17, 2015 5:03 PM
To: openstack-dev@lists.openstack.org
Subject: [openstack-dev] [Congress] Congress Usecases VM

Hi All,

I have put my VM (virtualbox) at:

http://paloaltan.net/Congress/Congress_Usecases_SEPT_17_2015.ova

I usually run this on a macbook air – but it should work on other platfroms as 
well. I chose virtualbox since it is free.

Please send me your usecases – I can incorporate in the VM and send you an 
updated image. Please take a look at the structure I have in place for the 
first usecase; would prefer it be the same for other usecases. (However I am 
still open to suggestions for changes)

Thanks,

-Shiv

___

Re: [openstack-dev] [Congress] hands on lab

[Alex Yip]

Thanks for the feedback David!

This VM does use the network, but only so that you can use the browser to 
connect to Horizon, and your own terminal to connect via SSH.  That is 
actually, not necessary, but it does make the user experience more responsive 
and makes cutting and pasting work.

I don't know why you encountered the error in your browser.  Were you able to 
debug that problem at all?

thanks, Alex


From: KARR, DAVID 
Sent: Monday, September 21, 2015 1:50 PM
To: Alex Yip
Subject: RE: [Congress] hands on lab

Alex, I'm going to step through this.  I'll mention some issues or questions 
here.

I imported this OVA on a CentOS7 laptop, and when I tried to start it, it 
failed with:

"Could not start the machine devstack-congress
because the following physical network interfaces
were not found:

en0: Wi-Fi (AirPort)(adapter 1)

..."

Note that this laptop was not connected to the internet at this point in time.

I then connected it to a Wifi hotspot (all I have right now is my cell phone as 
a hotspot) and reopened it, which succeeded.  If a network connection is 
required, the doc should probably mention this.

I then was able to ssh into the VM, using the ip I got from ifconfig in the VM, 
which was 192.168.122.1.  However, when I tried "You can also point your 
browser to the IP address to use Horizon on the devstack VM", by trying to 
visit 
"https://urldefense.proofpoint.com/v2/url?u=http-3A__192.168.122.1&d=BQIFAg&c=Sqcl0Ez6M0X8aeM67LKIiDJAXVeAw-YihVMNtXt-uEs&r=djA1lFdIf0--GIJ_8gr44Q&m=02Z4gmeF5zku2_PehZe2cUAOGiPmJAe0gxBmxx34Xrg&s=rlCFKv9aIfiM-L0tHfhNdtFtodSzVee_6JiOqf3_wcc&e=
 " (or 
"https://urldefense.proofpoint.com/v2/url?u=https-3A__192.168.122.1&d=BQIFAg&c=Sqcl0Ez6M0X8aeM67LKIiDJAXVeAw-YihVMNtXt-uEs&r=djA1lFdIf0--GIJ_8gr44Q&m=02Z4gmeF5zku2_PehZe2cUAOGiPmJAe0gxBmxx34Xrg&s=YoPOPcZQR31zL_iE2RuzhNZuqqgzrMZLWrVInhNrppk&e=
 " or 
"https://urldefense.proofpoint.com/v2/url?u=https-3A__192.168.122.1_dashboard&d=BQIFAg&c=Sqcl0Ez6M0X8aeM67LKIiDJAXVeAw-YihVMNtXt-uEs&r=djA1lFdIf0--GIJ_8gr44Q&m=02Z4gmeF5zku2_PehZe2cUAOGiPmJAe0gxBmxx34Xrg&s=L68PZtLGvb8HCd4A63M26pUfrv8UUamOMq-c2RxIxrI&e=
 ") from the sam
 e laptop, it either failed immediately or eventually timed out.

I haven't yet ran the "rejoin-stack.sh" script.

> -Original Message-
> From: Alex Yip [mailto:a...@vmware.com]
> Sent: Thursday, September 17, 2015 6:04 PM
> To: OpenStack Development Mailing List (not for usage questions)
> Subject: [openstack-dev] [Congress] hands on lab
>
> Hi all,
> I have created a VirtualBox VM that matches the Vancouver handson-
> lab here:
>
> https://urldefense.proofpoint.com/v2/url?u=https-3A__drive.google.com_file_d_0B94E7u1TIA8oTEdOQlFERkFwMUE_view-3Fu&d=BQIFAg&c=Sqcl0Ez6M0X8aeM67LKIiDJAXVeAw-YihVMNtXt-uEs&r=djA1lFdIf0--GIJ_8gr44Q&m=02Z4gmeF5zku2_PehZe2cUAOGiPmJAe0gxBmxx34Xrg&s=c3AoOYZEQmPPjdVclbwbOPILxWvEwyc0S9YVTKCNTHE&e=
> sp=sharing
>
> There's also an updated instruction document here:
>
> https://urldefense.proofpoint.com/v2/url?u=https-3A__docs.google.com_document_d_1ispwf56bX8sy9T0KZyosdHrSR9WHEVA&d=BQIFAg&c=Sqcl0Ez6M0X8aeM67LKIiDJAXVeAw-YihVMNtXt-uEs&r=djA1lFdIf0--GIJ_8gr44Q&m=02Z4gmeF5zku2_PehZe2cUAOGiPmJAe0gxBmxx34Xrg&s=4cWofXZgZ6YTIXb0FRNFFWa_89Gn0X5b5xz1E1YdLuM&e=
> 1oGEIYA22Orw/pub
>
> If you have some time, please try it out to see if it all works as
> expected.
> thanks, Alex
>
> ___
> ___
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: OpenStack-dev-
> requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [Congress] hands on lab

[Alex Yip]

Hi all,
I have created a VirtualBox VM that matches the Vancouver handson-lab here:

https://drive.google.com/file/d/0B94E7u1TIA8oTEdOQlFERkFwMUE/view?usp=sharing

There's also an updated instruction document here:

https://docs.google.com/document/d/1ispwf56bX8sy9T0KZyosdHrSR9WHEVA1oGEIYA22Orw/pub

If you have some time, please try it out to see if it all works as expected.
thanks, Alex

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Congress] CLI equivalent

[Alex Yip]

Try this:

openstack congress policy row list classification error?




From: Shiv Haris 
Sent: Wednesday, September 16, 2015 12:04 PM
To: OpenStack Development Mailing List (not for usage questions)
Subject: [openstack-dev] [Congress] CLI equivalent

Do we have a CLI way for  doing the equivalent of:

$ curl -X GET localhost:1789/v1/policies/classification/tables/error/rows
As described in the tutorial:

https://github.com/openstack/congress/blob/master/doc/source/tutorial-tenant-sharing.rst#listing-policy-violations

-Shiv

From: Tim Hinrichs [mailto:t...@styra.com]
Sent: Thursday, September 10, 2015 8:41 AM
To: OpenStack Development Mailing List (not for usage questions)
Subject: [openstack-dev] [Congress] Ending feature freeze

Hi all,

We're now finished with feature freeze.  We have our first release candidate 
and the stable/liberty branch.  So master is once again open for new features.  
Couple of things to note:

1. Documentation.  We should also look through the docs and update them.  
Documentation is really important.  There's one doc patch not yet merged, so be 
sure to pull that down before editing.  That patch officially deprecates a 
number of API calls that don't make sense for the new distributed architecture. 
 If you find places where we don't mention the deprecation, please fix that.

https://review.openstack.org/#/c/220707/

2. Bugs.  We should still all be manually testing, looking for bugs, and fixing 
them.  This will be true especially as other projects change their clients, 
which as we've seen can break our datasource drivers.

All bug fixes first go into master, and then we cherry-pick to stable/liberty.  
Once you've patched a bug on master and it's been merged, you'll create another 
change for your bug-fix and push it to review.  Then one of the cores will 
+2/+1 it (usually without needing another formal round of reviews).  Here's the 
procedure.

// pull down the latest changes for master
$ git checkout master
$ git pull

// create a local branch for stable/liberty and switch to it
$ git checkout origin/stable/liberty -b stable/liberty

// cherry-pick your change from master onto the local stable/liberty
// The -x records the original  in the commit msg
$ git cherry-pick -x 

// Push to review and specify the stable/liberty branch.
// Notice in gerrit that the branch is stable/liberty, not master
$ git review stable/liberty

// Once your change to stable/liberty gets merged, fetch all the new
// changes.

// switch to local version of stable/liberty
$ git checkout stable/liberty

// fetch all the new changes to all the branches
$ git fetch origin

// update your local branch
$ git rebase origin/stable/liberty

Tim





__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Congress] hosts table of nova datasource driver

[Alex Yip]

Hi Masahito,

Thanks for the reminder.  We do intend to keep the hosts table.  The code is 
commented out only because there is a bug that keeps the driver from working 
with the hosts table right now.  I don't know if anyone is looking into it, but 
I just created a bug report for it:

https://bugs.launchpad.net/congress/+bug/1441778

thanks, Alex


From: Masahito MUROI 
Sent: Tuesday, April 7, 2015 11:11 PM
To: openstack-dev@lists.openstack.org
Subject: [openstack-dev] [Congress] hosts table of nova datasource driver

Hi, congress folks

I'm new to congress and I'd like to use hosts table of nova datasource
driver in my usecase.  I, however, found out the hosts table is
commented out in current master.  Is there any reason to comment out it?
or will it be removed from an official datasource in the future?

best regard,
masa

--
室井 雅仁(Masahito MUROI)
Software Innovation Center, NTT
Tel: +81-422-59-4539


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Congress] Info on Currently Supported Data Drivers

[Alex Yip]

Hi Bryan,

The drive source code is the best documentation that describes the table and 
row level? detail.

The translator description in the source code sums up the tables an drivers in 
a succinct way.  If you need any additional clarification, let me know.

 - Alex



From: Bryan Sullivan 
Sent: Tuesday, March 24, 2015 12:13 PM
To: openstack-dev@lists.openstack.org
Subject: [openstack-dev] [Congress] Info on Currently Supported Data Drivers

Hi, I have a question about where to find the complete set of currently 
supported data drivers (with table/row details) for Congress. The info at 
http://congress.readthedocs.org/en/latest/cloudservices.html#drivers
 seems to cover only Nova and Neutron.  I'm sure that I can pull this info from 
the source at 
https://github.com/stackforge/congress/tree/master/congress/datasources
 but I was looking for any documentation on the current state of that code's 
design. The existing blueprints (e.g. at 
https://github.com/stackforge/congress-specs)
 do not go into this detail AFAICT.

Also, since I'm looking into how the current supported Congress tables will 
support use cases of the OPNFV Copper project [1], it will be also helpful to 
know where I would look for the set of potential policy-related data items 
inside OpenStack docs/code for the various components. If I find a gap against 
a use case, I would want to be able to specify related patches (e.g. where data 
source driver extensions should get the additional data) so I need to know 
where the data comes from (and what's available) inside OpenStack.

[1] https://wiki.opnfv.org/copper/use_cases

Thanks for your help!
Bryan





__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Congress] data-source renovation

[Alex Yip]

Hi all,

I favor the first approach because it solves the usability problem of wide 
tables without limiting Congress' ability to use wide tables, or adding extra 
complexity.

There are legitimate uses for wide tables, so Congress should be able to 
support them.  For example, Congress will need to support very large data 
sources in the future (TB in size).  It is best if Congress uses those 
databases in place, without creating a local copy of the database, so 
supporting wide tables and making them easy to use in the policy language will 
be a win for the future.

For Con (i) (we will need to invert the preprocessor when showing 
rules/traces/etc. to the user), we can keep the translated policies hidden from 
the user.  The user should only see policies that he wrote.

For Con (ii) (a layer of translation makes debugging difficult), the 
translation layer would be akin to a C preprocessor.  It will be possible to 
match up items on both sides of the translation layer.

- Alex


> Option 2 looks like a better idea keeping in mind the data model
> consistency with Neutron/Nova.
> Could we write something similar to a view which becomes a layer on top if
> this data model?


From: Tim Hinrichs
Sent: Tuesday, July 29, 2014 3:03 PM
To: openstack-dev@lists.openstack.org
Cc: Alex Yip
Subject: [Congress] data-source renovation

Hi all,

As I mentioned in a previous IRC, when writing our first few policies I had 
trouble using the tables we currently use to represent external data sources 
like Nova/Neutron.

The main problem is that wide tables (those with many columns) are hard to use. 
 (a) it is hard to remember what all the columns are, (b) it is easy to 
mistakenly use the same variable in two different tables in the body of the 
rule, i.e. to create an accidental join, (c) changes to the datasource drivers 
can require tedious/error-prone modifications to policy.

I see several options.  Once we choose something, I’ll write up a spec and 
include the other options as alternatives.


1) Add a preprocessor to the policy engine that makes it easier to deal with 
large tables via named-argument references.

Instead of writing a rule like

p(port_id, name) :-
neutron:ports(port_id, addr_pairs, security_groups, extra_dhcp_opts, 
binding_cap, status, name, admin_state_up, network_id, tenant_id, binding_vif, 
device_owner, mac_address, fixed_ips, router_id, binding_host)

we would write

p(id, nme) :-
neutron:ports(port_id=id, name=nme)

The preprocessor would fill in all the missing variables and hand the original 
rule off to the Datalog engine.

Pros: (i) leveraging vanilla database technology under the hood
  (ii) policy is robust to changes in the fields of the original data b/c 
the Congress data model is different than the Nova/Neutron data models
Cons: (i) we will need to invert the preprocessor when showing 
rules/traces/etc. to the user
  (ii) a layer of translation makes debugging difficult

2) Be disciplined about writing narrow tables and write 
tutorials/recommendations demonstrating how.

Instead of a table like...
neutron:ports(port_id, addr_pairs, security_groups, extra_dhcp_opts, 
binding_cap, status, name, admin_state_up, network_id, tenant_id, binding_vif, 
device_owner, mac_address, fixed_ips, router_id, binding_host)

we would have many tables...
neutron:ports(port_id)
neutron:ports.addr_pairs(port_id, addr_pairs)
neutron:ports.security_groups(port_id, security_groups)
neutron:ports.extra_dhcp_opts(port_id, extra_dhcp_opts)
neutron:ports.name(port_id, name)
...

People writing policy would write rules such as ...

p(x) :- neutron:ports.name(port, name), ...

[Here, the period e.g. in ports.name is not an operator--just a convenient way 
to spell the tablename.]

To do this, Congress would need to know which columns in a table are sufficient 
to uniquely identify a row, which in most cases is just the ID.

Pros: (i) this requires only changes in the datasource drivers; everything else 
remains the same
  (ii) still leveraging database technology under the hood
  (iii) policy is robust to changes in fields of original data
Cons: (i) datasource driver can force policy writer to use wide tables
  (ii) this data model is much different than the original data models
  (iii) we need primary-key information about tables

3) Enhance the Congress policy language to handle objects natively.

Instead of writing a rule like the following ...

p(port_id, name, group) :-
neutron:ports(port_id, addr_pairs, security_groups, extra_dhcp_opts, 
binding_cap, status, name, admin_state_up, network_id, tenant_id, binding_vif, 
device_owner, mac_address, fixed_ips, router_id, binding_host),
neutron:ports.security_groups(security_group, group)

we would write a rule such as
p(port_id, name) :-
neutron:ports(port),
port.name(name),
port.id(port_id),
port.security_groups(group)

The big difference here i