[389-devel] Review Request for test cases

2015-08-06 Thread Amita Sharma

Hi All,

I have automated few test cases for 
https://fedorahosted.org/389/attachment/ticket/47910/ .
Here is my patch :: 
https://fedorahosted.org/389/attachment/ticket/47910/0001-Ticket-47910-allow-logconv.pl-S-E-switches-to-work-e.2.patch


I request for your valuable feedback.

Thanks  Regards,
Amita
--
389-devel mailing list
389-devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/389-devel

[389-devel] 48239 lib389 Please review fix for un-allocated prefix in

2015-08-06 Thread William Brown
https://fedorahosted.org/389/ticket/48239

https://fedorahosted.org/389/attachment/ticket/48239/0001-Fix-for-prefix-allocat
ion-of-un-initialised-dirsrv-o.patch


-- 
William Brown will...@blackhats.net.au
--
389-devel mailing list
389-devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/389-devel

Re: [389-devel] Instance discovery tools in lib389

2015-08-06 Thread Mark Reynolds

Hi William,

On 08/06/2015 12:54 AM, William Brown wrote:

Hi,

I have been working on lib389 recently to bring in a number of the tools I have
developed professionally into the project so that others can benefit from this.
As you may have seen I've already submitted a few patches related to this.

I've done some of the api changes needed, but now I would like to make it a bit
easier to add command line tools that can utilise lib389.

This is right inline with what we would like to do as well.


I would like to add a tools directory to lib389 with a number of the cli tools
and helpers that I have written [1]. Is this the best location for these tools?
If there are really simple, maybe they could just be added to the 
tools.py/utils.py files?  More on this below...

Some of them could arguably go into the main line 389ds codebase, but I want to
use lib389 to back them.

This really depends on the tool, but see below...


Second, with these tools I don't want them to be complex.
+1  For lib389 inclusion they should be very simple, so that other tools 
can utilize them.

For example, I would
like a tool that can show what attribute can be held by what objectClass. I
would want the calling syntax to be as easy as:

attribute_query.py -i instance name attribute name

In order to achieve this there are a few changes I want to make.

First, is that there is already a DirSrv.list() function. This works well to
list instances on the system, but it's missing the instance name, so I want to
add this to the dictionary that is returned.

Second, I want to make it possible to initialise the DirSrv object from an
instance directory returned by the list() function. Normally this is done
through the allocate() function to establish the details of the connection, at
the moment the instance function doesn't return enough data to support this.

Either, I can add in functionality to the list() function to return more data
about each instance, possibly enough to just take the result from list() and put
it in as allocate(args) and have a working connection. Alternately I write a new
allocate_from_instance() (or some other name ...?) function that is able to do
the discovery of the needed data, and then call allocate.

Which would be the preferred method.
Both.  Expand the instance dictionary(your proposal could be useful for 
many other tasks/tools), and create the new allocate_from_instance 
function.


Comments and advice is appreciated.


[1] The tools I want to add are:
* List all known system instances
* schema query tools (list of object classes, attributes, query which
objectclasses can take which attribute)
This should really be added to schema.py, or at least the core 
functionality added to schema.py, then build your tool off of that?

* aci testing and manipulation tools. For example, does aci X do what I
suspect.
* aci inversion tool. convert != aci's into == aci's to help conversion to white
lists.
Maybe these aci tools could be partially integrated into a new acl.py 
file?  Kind of like what I suggested with the schema tools.



* Probably many more that I haven't thought of yet to help make administration
of a 389ds instance simpler.


Ultimately, I would like to see some of your new functionally go into 
the existing core module, and then write your tools based off of those.  
This way it can be leveraged outside of your specific task tools - for 
use in other new/future tools.  Of course I am specualting a bit, 
because I have not seen what you are working on yet.


Perhaps others on this list might have some opinions  thoughts as well?

In the meantime I would like to see what you are working on, and we can 
discuss this together and work towards an ideal approach.


Thanks again for your interest and contribution to the lib389 project!

Mark




--
389-devel mailing list
389-devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/389-devel

Re: [389-devel] please review: Ticket 47703 - Remove search limit for ACI group evaluation

2015-08-06 Thread William Brown
On Thu, 2015-08-06 at 16:13 -0400, Mark Reynolds wrote:
 https://fedorahosted.org/389/ticket/47703
 
 https://fedorahosted.org/389/attachment/ticket/47703/0001-Ticket-47703-remove-
 search-limit-for-aci-group-evalu.patch
 --
 389-devel mailing list
 389-devel@lists.fedoraproject.org
 https://admin.fedoraproject.org/mailman/listinfo/389-devel

I'm having a read through of this to try and understand more about this, and
hopefully to help with your review.

Why was the search limit added initially into the aci plugin? By deleting this
code, is there some other edge case that it may cause?

Would it be possible to make a unit test case for this. If you set the sizelimit
in cn=config to be low, say 5, you could easily make a group that has more
members, and then evaluate aci behaviour in a unit test?

If you are busy perhaps that's something I could knock up and test if you were
happy for me to do so.

Sincerely,

-- 
William Brown will...@blackhats.net.au
--
389-devel mailing list
389-devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/389-devel

Re: [389-devel] Instance discovery tools in lib389

2015-08-06 Thread William Brown

  I've done some of the api changes needed, but now I would like to make it a 
  bit
  easier to add command line tools that can utilise lib389.
 This is right inline with what we would like to do as well.

Glad to hear it. 

  
  I would like to add a tools directory to lib389 with a number of the cli 
  tools
  and helpers that I have written [1]. Is this the best location for these 
  tools?
 If there are really simple, maybe they could just be added to the 
 tools.py/utils.py files?  More on this below...

Yes and no. See below, as I think I explain a bit better what I'm working
towards.

  Some of them could arguably go into the main line 389ds codebase, but I 
  want 
  to
  use lib389 to back them.
 This really depends on the tool, but see below...
  
  Second, with these tools I don't want them to be complex.
 +1  For lib389 inclusion they should be very simple, so that other tools 
 can utilize them.

Glad that we agree on this.


  Either, I can add in functionality to the list() function to return more 
  data
  about each instance, possibly enough to just take the result from list() 
  and 
  put
  it in as allocate(args) and have a working connection. Alternately I write 
  a 
  new
  allocate_from_instance() (or some other name ...?) function that is able to 
  do
  the discovery of the needed data, and then call allocate.
  
  Which would be the preferred method.
 Both.  Expand the instance dictionary(your proposal could be useful for 
 many other tasks/tools), and create the new allocate_from_instance 
 function.

Well, there is no point doing both.

I was either going to expand the instance dict to include all the SER_*
attributes, so that you could literally pass it straight to allocate().

Or, you have just the instance name added to the instance dict, then you have
allocate_from_instance (Which also, I don't like that name even though I came up
with it. It's confusing :S ) which will then discover all the SER_* values, then
pass them all to allocate() anyway.

This is why I think we should choose on way or the other. My preference is
actually the former, as we avoid a confusing function name, and it makes the
value of the list() function greater. You could do:

insts = ds.list(all=True)
someinst = insts[0]
ds.allocate(someinst)

Or even allocate a new DirSrv, etc. 

  
  Comments and advice is appreciated.
  
  
  [1] The tools I want to add are:
  * List all known system instances
  * schema query tools (list of object classes, attributes, query which
  objectclasses can take which attribute)
 This should really be added to schema.py, or at least the core 
 functionality added to schema.py, then build your tool off of that?

 * aci testing and manipulation tools. For example, does aci X do what I
  suspect.
  * aci inversion tool. convert != aci's into == aci's to help conversion to 
  white
  lists.
 Maybe these aci tools could be partially integrated into a new acl.py 
 file?  Kind of like what I suggested with the schema tools.
  * Probably many more that I haven't thought of yet to help make 
  administration
  of a 389ds instance simpler.
 
 Ultimately, I would like to see some of your new functionally go into 
 the existing core module, and then write your tools based off of those.  
 This way it can be leveraged outside of your specific task tools - for 
 use in other new/future tools.  Of course I am specualting a bit, 
 because I have not seen what you are working on yet.

You have already seen my work, and acked most of the functionality into the core
modules. Additionally the aci.py module is waiting in your inbox for informal
review. 

For example: 

https://fedorahosted.org/389/ticket/48236 - Will be used in conjunction with
aci.py to evaluate and test aci functionality is working as expected, both in
the aci evaluation engine in 389ds, but also from a semantic point of view of
user acis

https://fedorahosted.org/389/ticket/48238 - Is all the functionality needed to
make the schema query tool I was discussing before.


My goal was to always add functionality into the library, then make the most
minimal command line tool possible to just stich it all together. This way both
unit tests and possible cli tools can benefit from the work. So I think we
already have the same approach, I just maybe didn't communicate that very well.

 
 Perhaps others on this list might have some opinions  thoughts as well?
 
 In the meantime I would like to see what you are working on, and we can 
 discuss this together and work towards an ideal approach.

Sounds like a plan! 

 
 Thanks again for your interest and contribution to the lib389 project!

You're welcome. Thanks for your time reviewing my work and ideas.

-- 
William Brown will...@blackhats.net.au
--
389-devel mailing list
389-devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/389-devel

Re: [389-devel] Review Request for test cases

2015-08-06 Thread Mark Reynolds



On 08/06/2015 06:24 AM, Amita Sharma wrote:

Hi All,

I have automated few test cases for 
https://fedorahosted.org/389/attachment/ticket/47910/ .
Here is my patch :: 
https://fedorahosted.org/389/attachment/ticket/47910/0001-Ticket-47910-allow-logconv.pl-S-E-switches-to-work-e.2.patch


I request for your valuable feedback.

Hi Amita,

Thanks for writing a lib389 test!  I do have some comments, see below...

In function log_dir:

First, you are using the DATA directory for temporary storage, you 
should be using the TMP dir - this way the contents are removed for you 
before the next test runs.


ldif_file = topology.standalone.getDir(__file__, TMP_DIR) + 
ticket47910.ldif


-
Note, for future tests, if you need to store files permanently, they 
should be in subdirectories of DATA:


getDir(__file__, DATA_DIR) + /ticket47910/ticket47910.ldif
-

Next in log_dir

You are generating an ldif file, and then importing it for each test 
function.  This seems excessive for trying to generate logging. The only 
thing that is logged is the adding of the task entry.  So like 6 lines 
of logging.  It would be easier/faster to just do a single search.  You 
can also disable access log buffering so you don't have to wait for the 
logging to be written to disk.


There is a shortcut function for setting access log buffering:

topology.standalone.setAccessLogBuffering (False)

You still need to sleep for 1 second, but it's a lot better than 50 seconds.

The rest looks good.

Thanks,
Mark


Thanks  Regards,
Amita
--
389-devel mailing list
389-devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/389-devel


--
389-devel mailing list
389-devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/389-devel