Re: [Freeipa-devel] [PATCHES 0227-0229] Server upgrade: introduce ipa-server-upgrade command

2015-04-28 Thread Jan Cholasta 

Dne 29.4.2015 v 08:45 Martin Kosek napsal(a):

On 04/29/2015 07:34 AM, Jan Cholasta wrote:

Dne 27.4.2015 v 18:23 David Kupka napsal(a):

On 04/27/2015 04:45 PM, Martin Basti wrote:

On 27/04/15 13:38, Martin Basti wrote:

On 23/04/15 12:55, Martin Basti wrote:

On 21/04/15 10:31, Martin Basti wrote:

On 21/04/15 08:12, Jan Cholasta wrote:

Hi,

Dne 15.4.2015 v 16:26 Martin Basti napsal(a):

https://fedorahosted.org/freeipa/ticket/4904

Patches attached.

Also ipa-upgradeconfig part is called as a subprocess. This will be
removed after installer modifications.

This patch may cause temporal upgrade issues (corner cases), until
installer part will be finished.

If somebody will be hit by them, please use --skip-version-check for
ipactl and ipa-server-upgrade.


Regarding that option vs. --force: I think the common assumption is
that --force ignores *all* non-fatal errors, but you break that
assumption in ipactl. IMO --force should both ignore errors in
service startup *and* skip version check, and a new option should
be added to just ignore errors in service startup (e.g.
--ignore-service-failures).

Originally I used --force option to skip detection, but there was
objections against it on list.

However, to have option --force, which set true for both
--ignore-service-failures and --skip-version-check options, might be
better.



ipa-server-upgrade should probably also have --force, even if it
does the same thing as --skip-version-check, again because --force
is common.


This is a weird API:

+if data_upgrade.badsyntax:
+raise admintool.ScriptError(
+'Bad syntax detected in upgrade file(s).', 1)
+elif data_upgrade.upgradefailed:
+raise admintool.ScriptError('IPA upgrade failed.', 1)
+elif data_upgrade.modified:
+self.log.info('Data update complete')
+else:
+self.log.info('Data update complete, no data were
modified')

Why does not IPAUpgrade raise errors instead?


For historical reasons, I can investigate what would break this
change, I will send it in separate patch.


+class IPAVersionError(Exception):
+pass
+
+class PlatformMismatchError(IPAVersionError):
+pass
+
+class DataUpgradeRequiredError(IPAVersionError):
+pass
+
+class DataInNewerVersionError(IPAVersionError):
+pass

I don't like the "IPA" in "IPAVersionError", it does not tell you
much about what kind of version is that. Also data version errors
should only tell you what is wrong, not how you fix it. IMO better
names for these would be e.g. "UpgradeVersionError",
"UpgradePlatformError", "UpgradeDataOlderVersionError",
"UpgradeDataNewerVersionError". Similar for store_ipa_version and
check_ipa_version.


Ok.


Why is it not an error if there is no version in check_ipa_version?
IMO it should, even if you then ignore the exception most of the
time.

I can raise error in that case and ignore the exception.



Honza


Martin^2


Updated patches attached.




Updated patches attached

--
Martin Basti




Updated patch attached



Looks good to me and works as expected. Honza, are you OK with the patches?



Some nitpicks:

The command line tool class should be named "ServerUpgrade" rather than
"IPAServerUpgrade" for consistency with others.

The deprecated --debug option should not be used in new commands.


Why is --debug option deprecated? I thought we wanted to deprecate --verbose
option as --debug is used in most our CLI tools. Well, except ipa-ldap-updated
which for some reasons marks --debug as deprecated. It does not matter now,
given the command is removed/changed.


AdminTool provides --debug as a deprecated alias for --verbose when a 
subclass requests it. It seems the decision to deprecate --debug was 
already made back when AdminTool was introduced, so let's trust that 
decision.


--
Jan Cholasta

--
Manage your subscription for the Freeipa-devel mailing list:
https://www.redhat.com/mailman/listinfo/freeipa-devel
Contribute to FreeIPA: http://www.freeipa.org/page/Contribute/Code

Re: [Freeipa-devel] [PATCHES 0227-0229] Server upgrade: introduce ipa-server-upgrade command

2015-04-28 Thread Martin Kosek 
On 04/29/2015 07:34 AM, Jan Cholasta wrote:
> Dne 27.4.2015 v 18:23 David Kupka napsal(a):
>> On 04/27/2015 04:45 PM, Martin Basti wrote:
>>> On 27/04/15 13:38, Martin Basti wrote:
 On 23/04/15 12:55, Martin Basti wrote:
> On 21/04/15 10:31, Martin Basti wrote:
>> On 21/04/15 08:12, Jan Cholasta wrote:
>>> Hi,
>>>
>>> Dne 15.4.2015 v 16:26 Martin Basti napsal(a):
 https://fedorahosted.org/freeipa/ticket/4904

 Patches attached.

 Also ipa-upgradeconfig part is called as a subprocess. This will be
 removed after installer modifications.

 This patch may cause temporal upgrade issues (corner cases), until
 installer part will be finished.

 If somebody will be hit by them, please use --skip-version-check for
 ipactl and ipa-server-upgrade.
>>>
>>> Regarding that option vs. --force: I think the common assumption is
>>> that --force ignores *all* non-fatal errors, but you break that
>>> assumption in ipactl. IMO --force should both ignore errors in
>>> service startup *and* skip version check, and a new option should
>>> be added to just ignore errors in service startup (e.g.
>>> --ignore-service-failures).
>> Originally I used --force option to skip detection, but there was
>> objections against it on list.
>>
>> However, to have option --force, which set true for both
>> --ignore-service-failures and --skip-version-check options, might be
>> better.
>>
>>>
>>> ipa-server-upgrade should probably also have --force, even if it
>>> does the same thing as --skip-version-check, again because --force
>>> is common.
>>>
>>>
>>> This is a weird API:
>>>
>>> +if data_upgrade.badsyntax:
>>> +raise admintool.ScriptError(
>>> +'Bad syntax detected in upgrade file(s).', 1)
>>> +elif data_upgrade.upgradefailed:
>>> +raise admintool.ScriptError('IPA upgrade failed.', 1)
>>> +elif data_upgrade.modified:
>>> +self.log.info('Data update complete')
>>> +else:
>>> +self.log.info('Data update complete, no data were
>>> modified')
>>>
>>> Why does not IPAUpgrade raise errors instead?
>>>
>> For historical reasons, I can investigate what would break this
>> change, I will send it in separate patch.
>>>
>>> +class IPAVersionError(Exception):
>>> +pass
>>> +
>>> +class PlatformMismatchError(IPAVersionError):
>>> +pass
>>> +
>>> +class DataUpgradeRequiredError(IPAVersionError):
>>> +pass
>>> +
>>> +class DataInNewerVersionError(IPAVersionError):
>>> +pass
>>>
>>> I don't like the "IPA" in "IPAVersionError", it does not tell you
>>> much about what kind of version is that. Also data version errors
>>> should only tell you what is wrong, not how you fix it. IMO better
>>> names for these would be e.g. "UpgradeVersionError",
>>> "UpgradePlatformError", "UpgradeDataOlderVersionError",
>>> "UpgradeDataNewerVersionError". Similar for store_ipa_version and
>>> check_ipa_version.
>>>
>> Ok.
>>>
>>> Why is it not an error if there is no version in check_ipa_version?
>>> IMO it should, even if you then ignore the exception most of the
>>> time.
>> I can raise error in that case and ignore the exception.
>>>
>>>
>>> Honza
>>>
>> Martin^2
>>
> Updated patches attached.
>
>
>
 Updated patches attached

 -- 
 Martin Basti


>>>
>>> Updated patch attached
>>>
>>
>> Looks good to me and works as expected. Honza, are you OK with the patches?
>>
> 
> Some nitpicks:
> 
> The command line tool class should be named "ServerUpgrade" rather than
> "IPAServerUpgrade" for consistency with others.
> 
> The deprecated --debug option should not be used in new commands.

Why is --debug option deprecated? I thought we wanted to deprecate --verbose
option as --debug is used in most our CLI tools. Well, except ipa-ldap-updated
which for some reasons marks --debug as deprecated. It does not matter now,
given the command is removed/changed.

> 
> I would like to see --skip-version-check also in ipa-server-upgrade, for
> consistency with ipactl.
> 
> In the spec file ipa-server-upgrade is run with --quiet, so why redirect 
> stdout
> to /dev/null?

-- 
Manage your subscription for the Freeipa-devel mailing list:
https://www.redhat.com/mailman/listinfo/freeipa-devel
Contribute to FreeIPA: http://www.freeipa.org/page/Contribute/Code

Re: [Freeipa-devel] Suggestion for the A part of IPA

2015-04-28 Thread Martin Kosek 
On 04/28/2015 05:58 PM, Innes, Duncan wrote:
> Folks,
>  
> The A part of IPA has always been of great interest to me.  Our current
> IPA infrastructure works well at the I & P parts, giving us great
> failover abilities and connectivity through hardware firewalls without
> punching too many holes.

Good to hear :-) We recently also started investigating the Audit capabilities
for (notice I write "for" and not "in") IPA. You can check my initial nudge to
the freeipa-users list, which was unfortunately with no reply:

https://www.redhat.com/archives/freeipa-users/2015-March/msg00940.html

For the beginning, I would be interested for your use cases, if you are only
looking for a centralized log store, or you are also looking for more analytics
in the logs (like what API commands were run, user logins, etc.) or utilization
of the server core services (LDAP/Kerberos/DNS/...)

> Whilst the A part may not be solely about centralised logging, it's the
> thing I've been looking into recently.  To do this I've built a setup
> around the ELK stack using a pair of Logstash servers and an
> ElasticSearch cluster of 5 servers (overkill on the ES side perhaps, but
> this is proof of concept still).  To expand on this, I've been looking
> at running the Logstash serviceon each of our IPA servers as that gives
> us a failover pair in each part of our network.  The Logstash servers
> then connect to the ES cluster as non-data nodes.  Each client has an
> rsyslog7 (still using RHEL6 at the moment) config that writes sends the
> logs in JSON format with some extra bespoke fields added (such as
> Project, Environment, and Use to help us search better).  The sending is
> done in rsyslog's rather clunky failover method to the local pair of
> Logstash servers (with a third failover being to /dev/null).

Ah, so you are running Logstash service on each IPA service? Isn't that too
heavyweight? In our tests, we mostly simply wanted just configure rsyslog and
get the logs out of the server, to the centralized ELK/REK/EFK servers which
did the heavy lifting. After all, the IPA servers may be of different
environments (Fedora, RHEL, CentOS, ...) and with different versions of the log
processing software.

On the REK server (yes, we did not use logstash at the POC), we are able to
process the logs (make them structured), store and display them. This allows us
do searches like "list of admins which added users in the last month".

This if course required adding parsing rules to rsyslog to get the structure
out of the API logs. Are you using logstash for the parsing or did you not
start the parsing part yet?

> It struck me that this kind of setup might not be too far removed from
> some of the A part of IPA.

The centralized log processing itself is a too big task for IPA itself, it is
specialized on other things. But some integration should be added in time, I
agree. It may be minimal, from top of my head for example:

* Support of (rsyslog) configuration in ipa-client-install or ipa-server-install
* Providing the secure, GSSAPI-based log transfer to the IPA clients and
ELK/REK/EFK server
* Providing parsing templates for rsyslog or base queries for Kibana

> I'm not good at ASCII flowchart diagrams, so will leave it there for
> now.  The main point of this - does any of this idea sound reasonable to
> add in to FreeIPA?  To me it sounds like a good fit for getting (some)
> logging data back to a central point.
>  
> The Logstash indexers currently have a very low load (perhaps due to the
> incoming data already being JSON) and small memory footprint.  They run
> without issue on our IPA servers.  The ES nodes are different and I
> won't pretent to be any sort of expert in what they do.  They load up a
> bit when I shut 1 of them down, but that's the rebalancing happening.
>  
> Apologies if this is off topic, or wide of the mark.
>  
> Cheers
>  
> Duncan Innes
> 
> This message has been checked for viruses and spam by the Virgin Money email 
> scanning system powered by Messagelabs.
> 
> This e-mail is intended to be confidential to the recipient. If you receive a 
> copy in error, please inform the sender and then delete this message.
> 
> Virgin Money plc - Registered in England and Wales (Company no. 6952311). 
> Registered office - Jubilee House, Gosforth, Newcastle upon Tyne NE3 4PL. 
> Virgin Money plc is authorised by the Prudential Regulation Authority and 
> regulated by the Financial Conduct Authority and the Prudential Regulation 
> Authority.
> 
> The following companies also trade as Virgin Money. They are both authorised 
> and regulated by the Financial Conduct Authority, are registered in England 
> and Wales and have their registered office at Jubilee House, Gosforth, 
> Newcastle upon Tyne NE3 4PL: Virgin Money Personal Financial Service Limited 
> (Company no. 3072766) and Virgin Money Unit Trust Managers Limited (Company 
> no. 3000482).
> 
> For further details of Virgin Money group companies please visit our webs

Re: [Freeipa-devel] [PATCHES 0227-0229] Server upgrade: introduce ipa-server-upgrade command

2015-04-28 Thread Jan Cholasta 

Dne 27.4.2015 v 18:23 David Kupka napsal(a):

On 04/27/2015 04:45 PM, Martin Basti wrote:

On 27/04/15 13:38, Martin Basti wrote:

On 23/04/15 12:55, Martin Basti wrote:

On 21/04/15 10:31, Martin Basti wrote:

On 21/04/15 08:12, Jan Cholasta wrote:

Hi,

Dne 15.4.2015 v 16:26 Martin Basti napsal(a):

https://fedorahosted.org/freeipa/ticket/4904

Patches attached.

Also ipa-upgradeconfig part is called as a subprocess. This will be
removed after installer modifications.

This patch may cause temporal upgrade issues (corner cases), until
installer part will be finished.

If somebody will be hit by them, please use --skip-version-check for
ipactl and ipa-server-upgrade.


Regarding that option vs. --force: I think the common assumption is
that --force ignores *all* non-fatal errors, but you break that
assumption in ipactl. IMO --force should both ignore errors in
service startup *and* skip version check, and a new option should
be added to just ignore errors in service startup (e.g.
--ignore-service-failures).

Originally I used --force option to skip detection, but there was
objections against it on list.

However, to have option --force, which set true for both
--ignore-service-failures and --skip-version-check options, might be
better.



ipa-server-upgrade should probably also have --force, even if it
does the same thing as --skip-version-check, again because --force
is common.


This is a weird API:

+if data_upgrade.badsyntax:
+raise admintool.ScriptError(
+'Bad syntax detected in upgrade file(s).', 1)
+elif data_upgrade.upgradefailed:
+raise admintool.ScriptError('IPA upgrade failed.', 1)
+elif data_upgrade.modified:
+self.log.info('Data update complete')
+else:
+self.log.info('Data update complete, no data were
modified')

Why does not IPAUpgrade raise errors instead?


For historical reasons, I can investigate what would break this
change, I will send it in separate patch.


+class IPAVersionError(Exception):
+pass
+
+class PlatformMismatchError(IPAVersionError):
+pass
+
+class DataUpgradeRequiredError(IPAVersionError):
+pass
+
+class DataInNewerVersionError(IPAVersionError):
+pass

I don't like the "IPA" in "IPAVersionError", it does not tell you
much about what kind of version is that. Also data version errors
should only tell you what is wrong, not how you fix it. IMO better
names for these would be e.g. "UpgradeVersionError",
"UpgradePlatformError", "UpgradeDataOlderVersionError",
"UpgradeDataNewerVersionError". Similar for store_ipa_version and
check_ipa_version.


Ok.


Why is it not an error if there is no version in check_ipa_version?
IMO it should, even if you then ignore the exception most of the
time.

I can raise error in that case and ignore the exception.



Honza


Martin^2


Updated patches attached.




Updated patches attached

--
Martin Basti




Updated patch attached



Looks good to me and works as expected. Honza, are you OK with the patches?



Some nitpicks:

The command line tool class should be named "ServerUpgrade" rather than 
"IPAServerUpgrade" for consistency with others.


The deprecated --debug option should not be used in new commands.

I would like to see --skip-version-check also in ipa-server-upgrade, for 
consistency with ipactl.


In the spec file ipa-server-upgrade is run with --quiet, so why redirect 
stdout to /dev/null?


--
Jan Cholasta

--
Manage your subscription for the Freeipa-devel mailing list:
https://www.redhat.com/mailman/listinfo/freeipa-devel
Contribute to FreeIPA: http://www.freeipa.org/page/Contribute/Code

Re: [Freeipa-devel] [PATCH 0029] suppress errors arising from deleting non-existent files during client uninstall

2015-04-28 Thread Jan Cholasta 

Dne 28.4.2015 v 15:03 Martin Basti napsal(a):

On 27/04/15 10:33, Martin Babinsky wrote:

On 04/24/2015 03:19 PM, Martin Basti wrote:

On 20/04/15 10:58, Martin Babinsky wrote:

On 04/20/2015 10:32 AM, Martin Basti wrote:

On 17/04/15 14:11, Martin Babinsky wrote:

On 04/17/2015 12:41 PM, Martin Babinsky wrote:

On 04/17/2015 12:36 PM, Martin Basti wrote:

On 17/04/15 12:33, Martin Babinsky wrote:

On 04/17/2015 12:04 PM, Martin Basti wrote:

On 15/04/15 15:53, Martin Babinsky wrote:

On 04/14/2015 04:24 PM, Martin Basti wrote:

On 14/04/15 16:12, Martin Basti wrote:

On 14/04/15 14:25, Martin Babinsky wrote:

This patch addresses
https://fedorahosted.org/freeipa/ticket/4966

The noise during rollback/uninstall is caused mainly by
unsuccessful
attempts to remove files that do not exist anymore. These
errors
are
now logged at debug level and do not pop-up to stdout/stderr.




Hello, thank you for the patch.

1)
The option add_warning is quite unclear to me. It does not
show
warning but error. I suggest something like, show_hint,
show_user_action, or something show_additional_..., or
promt_manual_removal

Martin^2



Continue...

2)

 if file_exists(preferences_fname):
 try:
os.remove(preferences_fname)
 except OSError as e:
 log_file_removal_error(e,
preferences_fname,
True)

In this case file not found error should never happen.

Could you remove the 'if file_exists' part and handle just
exception?


I just reverted this bit to original form in order to not fix
something that isn't broken. Is that ok?

3)
this is inconsistent with change above, choose one style
please:

 if os.path.exists(ca_file):
 try:
 os.unlink(ca_file)
 except OSError, e:
 root_logger.error(
 "Failed to remove '%s': %s",
ca_file, e)

--
Martin Basti



Attaching updated patch.


thanks,

just one nitpick, can you move the new function into
installutils, it
can be used in different scripts not just in ipaclient.



I'm not sure if it is a good idea as installutils is a part for
freeipa-server package.

Placing it there would create an unnecessary dependency of
freeipa-client on freeipa-server because of a single function.


you are right, I do not why I thought that ipa-client-install uses
installutils.

ACK


self-NACK, I will try to rewrite the patch in a slightly less dumb
way.

Sorry for the confusion.



Attaching updated patch which does the same but using a wrapper
around
os.remove().

Jan suggested to keep the new function in 'ipa-client-install' and
move it around when we do installer re#$%@^ing.

Is that ok?


It looks better, ACK.


Jan NACKed your ACK.

Attaching updated patch.


Sorry, NACK

* Module ipa-client-install
ipa-client/ipa-install/ipa-client-install:791:
[E1121(too-many-function-args), uninstall] Too many positional arguments
for function call)
ipa-client/ipa-install/ipa-client-install:797:
[E1121(too-many-function-args), uninstall] Too many positional arguments
for function call)

consult with Honza if option which show prompt user to delete file
manually, should be there or not.



Updated patch attached.


ACK



Pushed to:
master: 98376589de9b33d7007c8d43366d26f3e3307662
ipa-4-1: b04435a0f5c63e18ec36f2d3e0849a6dee384589

--
Jan Cholasta

--
Manage your subscription for the Freeipa-devel mailing list:
https://www.redhat.com/mailman/listinfo/freeipa-devel
Contribute to FreeIPA: http://www.freeipa.org/page/Contribute/Code

Re: [Freeipa-devel] [PATCH 0014] emit a more helpful error messages when CA configuration fails

2015-04-28 Thread Jan Cholasta 

Dne 28.4.2015 v 15:03 Martin Basti napsal(a):

On 27/04/15 10:54, Martin Babinsky wrote:

On 04/24/2015 04:15 PM, Martin Basti wrote:

On 20/04/15 12:59, Martin Babinsky wrote:

On 04/17/2015 03:56 PM, Martin Babinsky wrote:

On 03/05/2015 01:11 PM, Martin Babinsky wrote:

https://fedorahosted.org/freeipa/ticket/4900



___
Freeipa-devel mailing list
Freeipa-devel@redhat.com
https://www.redhat.com/mailman/listinfo/freeipa-devel



Nobody to review this?



Attaching updated patches, one for ipa-4-1 (no DogtagInstance) and one
for master.




Hello, thank for patches:

1)
why is there

+PKI_UNINSTALL_LOG = paths.PKI_CA_UNINSTALL_LOG

I cannot find it used in patches?


Martin^2

--
Martin Basti


That was likely only my oversight. Attaching updated patches.


ACK



Pushed to:
master: a1f91247ccf69a60d1e18942e6697f45b951fe4b
ipa-4-1: 04f5842784e06f7e3f973b534d34d08a74a7

(Shamelessly added myself as a reviewer, since I did an offline review 
which lead to the changes done between the first and second generation 
of the patches.)


--
Jan Cholasta

--
Manage your subscription for the Freeipa-devel mailing list:
https://www.redhat.com/mailman/listinfo/freeipa-devel
Contribute to FreeIPA: http://www.freeipa.org/page/Contribute/Code

Re: [Freeipa-devel] Suggestion for the A part of IPA

2015-04-28 Thread Adam Young 

On 04/28/2015 11:58 AM, Innes, Duncan wrote:

Folks,
The A part of IPA has always been of great interest to me.  Our 
current IPA infrastructure works well at the I & P parts, giving us 
great failover abilities and connectivity through hardware firewalls 
without punching too many holes.
Whilst the A part may not be solely about centralised logging, it's 
the thing I've been looking into recently.  To do this I've built a 
setup around the ELK stack using a pair of Logstash servers and an 
ElasticSearch cluster of 5 servers (overkill on the ES side perhaps, 
but this is proof of concept still).  To expand on this, I've been 
looking at running the Logstash serviceon each of our IPA servers as 
that gives us a failover pair in each part of our network.  The 
Logstash servers then connect to the ES cluster as non-data nodes. 
Each client has an rsyslog7 (still using RHEL6 at the moment) config 
that writes sends the logs in JSON format with some extra bespoke 
fields added (such as Project, Environment, and Use to help us search 
better).  The sending is done in rsyslog's rather clunky failover 
method to the local pair of Logstash servers (with a third failover 
being to /dev/null).

I think  I am in alignment with what you are saying.

I like  rsyslogd as the basic "ship the log off the server" tool. Let's 
use what the platform support first natively and formost;  We want 
something native, not Ruby, not even Python if we can avoid it, for the 
normal case.  Bumping up to logstash for more complex host-side rules 
might be fine.  Remember, the Hosts side of integration with FreeIPA is 
sssd.


Logstash can be the server side of the audit collection as well, and 
then it puts fewer demands on the server.


We need to ensure that the audit data can be sent over a GSSAPI 
protected pathway.



On the IPA side, I would think we would register the audit server as a 
host, and have  specific service entires for the protocols supported.



Would you see IPA owning the audit server, or just integrating in with 
an existing one?


I don't think the IPA server itself should be the ELK server for obvious 
reasons. I would love to see the ELK server supported along the lines of 
how we do a replica setup.




It struck me that this kind of setup might not be too far removed from 
some of the A part of IPA.
I'm not good at ASCII flowchart diagrams, so will leave it there for 
now.  The main point of this - does any of this idea sound reasonable 
to add in to FreeIPA?  To me it sounds like a good fit for getting 
(some) logging data back to a central point.
The Logstash indexers currently have a very low load (perhaps due to 
the incoming data already being JSON) and small memory footprint.  
They run without issue on our IPA servers.  The ES nodes are different 
and I won't pretent to be any sort of expert in what they do.  They 
load up a bit when I shut 1 of them down, but that's the rebalancing 
happening.

Apologies if this is off topic, or wide of the mark.
Cheers
Duncan Innes

This message has been checked for viruses and spam by the Virgin Money 
email scanning system powered by Messagelabs.


This e-mail is intended to be confidential to the recipient. If you 
receive a copy in error, please inform the sender and then delete this 
message.


Virgin Money plc - Registered in England and Wales (Company no. 
6952311). Registered office - Jubilee House, Gosforth, Newcastle upon 
Tyne NE3 4PL. Virgin Money plc is authorised by the Prudential 
Regulation Authority and regulated by the Financial Conduct Authority 
and the Prudential Regulation Authority.


The following companies also trade as Virgin Money. They are both 
authorised and regulated by the Financial Conduct Authority, are 
registered in England and Wales and have their registered office at 
Jubilee House, Gosforth, Newcastle upon Tyne NE3 4PL: Virgin Money 
Personal Financial Service Limited (Company no. 3072766) and Virgin 
Money Unit Trust Managers Limited (Company no. 3000482).


For further details of Virgin Money group companies please visit our 
website at virginmoney.com





-- 
Manage your subscription for the Freeipa-devel mailing list:
https://www.redhat.com/mailman/listinfo/freeipa-devel
Contribute to FreeIPA: http://www.freeipa.org/page/Contribute/Code

Re: [Freeipa-devel] [Freeipa-users] FreeIPA and sambaPwdLastSet

2015-04-28 Thread Alexander Bokovoy 

Sending my answer to the list too.

On Tue, 28 Apr 2015, Alexander Bokovoy wrote:

On Tue, 28 Apr 2015, Christopher Lamb wrote:


Hi All

I wish to pick your brains on the attribute sambaPwdLastSet

We have a newly setup FreeIPA 4.1.0, with users and groups migrated from an
old 3.0.0 instance.

We are also running Samba to share files to Windows and OSX users. This
means that all the FreeIPA user accounts have the attribute
sambaPwdLastSet.

If this has the value 0, our users cannot map Samba shares, so we need to
make sure the value is a positive integer.

In an attempt to do this, I modified user.py, adding the attribute to the
takes_params for the class user as follows:

class user(LDAPObject):
 . . .
 takes_params = (
. . .
   Int('sambapwdlastset?',
  label=_('sambaPwdLastSet'),
  doc=_('Date as an integer when the samba password was last set'
),
  default=1,
  autofill=True,
  ),
  . . .

This works fine if I create a user via the CLI.

However if I create a user via the Web UI, or use the Web UI to reset a
user's password, then the attribute sambaPwdLastSet is set to zero.

So what scripts do I need to change to make sure the Web UI sets
sambaPwdLast Set to a positive value? (I don't want to run ldapmodify
scripts, or have to use Apache Directory Studio to hack the db..)

Or is there an altogether better approach to handling this field?

Yes, there is.

Given that you are running FreeIPA 4.1, you now can use SSSD as your
libwbclient provider to be able to run Samba on IPA client against IPA
database. There will be no dependency on sambaPwdLastSet anymore.

See
http://www.freeipa.org/page/Howto/Integrating_a_Samba_File_Server_With_IPA

This approach requires Fedora 21 or RHEL 7.1 / CentOS 7.1 on the IPA
client. It does not work though with non-Kerberos (NTLM) logins.

However, if you insist on using sambaPwdLastSet attribute, then user
password change rule is applying:

- if admin changes user password, sambaPwdLastSet is cleared to 0 to
 force users to change their passwords also via Samba

If user changes the password him/herself, sambaPwdLastSet is set to the
current time (i.e. not 0).

This really goes into enforcing privacy of user passwords -- if admins
change user passwords, the password is not really secret anymore and
cannot be considered secure, so it is only used once.

See also https://www.freeipa.org/page/Self-Service_Password_Reset and
https://www.freeipa.org/page/New_Passwords_Expired

--
/ Alexander Bokovoy


--
/ Alexander Bokovoy

--
Manage your subscription for the Freeipa-devel mailing list:
https://www.redhat.com/mailman/listinfo/freeipa-devel
Contribute to FreeIPA: http://www.freeipa.org/page/Contribute/Code

Re: [Freeipa-devel] [PATCH 0046] Remove unneeded --ip-address option in ipa-adtrust-install

2015-04-28 Thread Martin Basti 

On 28/04/15 15:36, Gabe Alford wrote:
On Tue, Apr 28, 2015 at 7:18 AM, Martin Basti > wrote:


On 27/04/15 15:02, Gabe Alford wrote:

Hello,

Fix for https://fedorahosted.org/freeipa/ticket/4575

Thanks,

Gabe



Hello,

thank you for your patch, looks good.

However, IMO you can remove hostaddr variable as well, it was used
for validate IP address.
I could not find it used anywhere, did you?


Removed. It still seemed like it could be a logical check to have to 
me which is why I kept it in the first place.

Updated patch attached.

Martin^2

-- 
Martin Basti





Thank you, ACK!

This check is not needed anymore, it will not work with dualstack, and 
adtrustisntance is installed on IPA server which must have resolvable IP 
address.


--
Martin Basti

-- 
Manage your subscription for the Freeipa-devel mailing list:
https://www.redhat.com/mailman/listinfo/freeipa-devel
Contribute to FreeIPA: http://www.freeipa.org/page/Contribute/Code

[Freeipa-devel] Suggestion for the A part of IPA

2015-04-28 Thread Innes, Duncan 
Folks,
 
The A part of IPA has always been of great interest to me.  Our current
IPA infrastructure works well at the I & P parts, giving us great
failover abilities and connectivity through hardware firewalls without
punching too many holes.
 
Whilst the A part may not be solely about centralised logging, it's the
thing I've been looking into recently.  To do this I've built a setup
around the ELK stack using a pair of Logstash servers and an
ElasticSearch cluster of 5 servers (overkill on the ES side perhaps, but
this is proof of concept still).  To expand on this, I've been looking
at running the Logstash serviceon each of our IPA servers as that gives
us a failover pair in each part of our network.  The Logstash servers
then connect to the ES cluster as non-data nodes.  Each client has an
rsyslog7 (still using RHEL6 at the moment) config that writes sends the
logs in JSON format with some extra bespoke fields added (such as
Project, Environment, and Use to help us search better).  The sending is
done in rsyslog's rather clunky failover method to the local pair of
Logstash servers (with a third failover being to /dev/null).
 
It struck me that this kind of setup might not be too far removed from
some of the A part of IPA.
 
I'm not good at ASCII flowchart diagrams, so will leave it there for
now.  The main point of this - does any of this idea sound reasonable to
add in to FreeIPA?  To me it sounds like a good fit for getting (some)
logging data back to a central point.
 
The Logstash indexers currently have a very low load (perhaps due to the
incoming data already being JSON) and small memory footprint.  They run
without issue on our IPA servers.  The ES nodes are different and I
won't pretent to be any sort of expert in what they do.  They load up a
bit when I shut 1 of them down, but that's the rebalancing happening.
 
Apologies if this is off topic, or wide of the mark.
 
Cheers
 
Duncan Innes

This message has been checked for viruses and spam by the Virgin Money email 
scanning system powered by Messagelabs.

This e-mail is intended to be confidential to the recipient. If you receive a 
copy in error, please inform the sender and then delete this message.

Virgin Money plc - Registered in England and Wales (Company no. 6952311). 
Registered office - Jubilee House, Gosforth, Newcastle upon Tyne NE3 4PL. 
Virgin Money plc is authorised by the Prudential Regulation Authority and 
regulated by the Financial Conduct Authority and the Prudential Regulation 
Authority.

The following companies also trade as Virgin Money. They are both authorised 
and regulated by the Financial Conduct Authority, are registered in England and 
Wales and have their registered office at Jubilee House, Gosforth, Newcastle 
upon Tyne NE3 4PL: Virgin Money Personal Financial Service Limited (Company no. 
3072766) and Virgin Money Unit Trust Managers Limited (Company no. 3000482).

For further details of Virgin Money group companies please visit our website at 
virginmoney.com-- 
Manage your subscription for the Freeipa-devel mailing list:
https://www.redhat.com/mailman/listinfo/freeipa-devel
Contribute to FreeIPA: http://www.freeipa.org/page/Contribute/Code

[Freeipa-devel] [PATCHES 0031-0032] set up a dedicated CCache file for Apache during install/upgrade

2015-04-28 Thread Martin Babinsky 
The attached patches address 
https://fedorahosted.org/freeipa/ticket/4973 and implement the solution 
proposed in Comment 2.


Please review the hell out of them.

--
Martin^3 Babinsky
From 6a6a4561f0faf817a027280bb93e74823416ed01 Mon Sep 17 00:00:00 2001
From: Martin Babinsky 
Date: Tue, 28 Apr 2015 16:24:02 +0200
Subject: [PATCH 1/2] HTTPInstance should set up dedicated CCache file for
 httpd

httpd service stores Kerberos credentials in kernel keyring which gets
destroyed and recreated during service install/upgrade. If the process is run
under SELinux context other than 'unconfined_t', the recreated keyring is
inaccessible to Apache. This patch enables HTTPInstance to set up a dedicated
CCache file for Apache to store credentials.

https://fedorahosted.org/freeipa/ticket/4973
---
 install/share/Makefile.am|  1 +
 install/share/httpd.service.template |  4 
 ipaserver/install/httpinstance.py| 18 ++
 3 files changed, 23 insertions(+)
 create mode 100644 install/share/httpd.service.template

diff --git a/install/share/Makefile.am b/install/share/Makefile.am
index ca6128e2911ab5c0a773dd553f8e67eab944f120..15f4a323d5a2df1af9522611b4453d1d9dae7c94 100644
--- a/install/share/Makefile.am
+++ b/install/share/Makefile.am
@@ -76,6 +76,7 @@ app_DATA =\
 	copy-schema-to-ca.py		\
 	sasl-mapping-fallback.ldif	\
 	schema-update.ldif		\
+	httpd.service.template	\
 	$(NULL)
 
 EXTRA_DIST =\
diff --git a/install/share/httpd.service.template b/install/share/httpd.service.template
new file mode 100644
index ..f990c92fb1f23f43dfa6df8795d471e0ea937c1d
--- /dev/null
+++ b/install/share/httpd.service.template
@@ -0,0 +1,4 @@
+.include $LIB_SYSTEMD_HTTPD_SERVICE
+
+[Service]
+Environment=KRB5CCNAME=/tmp/ipa-httpd.ccache
diff --git a/ipaserver/install/httpinstance.py b/ipaserver/install/httpinstance.py
index 18cf6bb1a55512f475bde62b2db7a775945a97ec..7568f4df239e0da3f09438021fa399cc4f841ed9 100644
--- a/ipaserver/install/httpinstance.py
+++ b/ipaserver/install/httpinstance.py
@@ -87,6 +87,8 @@ class HTTPInstance(service.Service):
 
 self.cert_nickname = cert_nickname
 self.ca_is_configured = True
+self.service_file = os.path.join(paths.ETC_SYSTEMD_SYSTEM_DIR,
+ 'httpd.service')
 
 subject_base = ipautil.dn_attribute_property('_subject_base')
 
@@ -109,6 +111,8 @@ class HTTPInstance(service.Service):
 DOMAIN=self.domain,
 AUTOREDIR='' if auto_redirect else '#',
 CRL_PUBLISH_PATH=dogtag.install_constants.CRL_PUBLISH_PATH,
+LIB_SYSTEMD_HTTPD_SERVICE=os.path.join(
+paths.LIB_SYSTEMD_SYSTEMD_DIR, 'httpd.service')
 )
 self.ca_file = ca_file
 if ca_is_configured is not None:
@@ -135,6 +139,8 @@ class HTTPInstance(service.Service):
 self.step("publish CA cert", self.__publish_ca_cert)
 self.step("creating a keytab for httpd", self.__create_http_keytab)
 self.step("clean up any existing httpd ccache", self.remove_httpd_ccache)
+self.step("creating custom systemd service file",
+  self.create_custom_service_file)
 self.step("configuring SELinux for httpd", self.configure_selinux_for_httpd)
 self.step("restarting httpd", self.__start)
 self.step("configuring httpd to start on boot", self.__enable)
@@ -191,6 +197,15 @@ class HTTPInstance(service.Service):
 http_fd.close()
 os.chmod(target_fname, 0644)
 
+def create_custom_service_file(self):
+service_file_template = os.path.join(ipautil.SHARE_DIR,
+ 'httpd.service.template')
+
+ipautil.copy_template_file(service_file_template,
+   self.service_file,
+   self.sub_dict)
+os.chmod(self.service_file, 0644)
+
 def change_mod_nss_port_from_http(self):
 # mod_ssl enforces SSLEngine on for vhost on 443 even though
 # the listener is mod_nss. This then crashes the httpd as mod_nss
@@ -432,6 +447,9 @@ class HTTPInstance(service.Service):
 if enabled:
 self.enable()
 
+# remove the custom service file
+installutils.remove_file(self.service_file)
+
 def stop_tracking_certificates(self):
 db = certs.CertDB(api.env.realm)
 db.untrack_server_cert(self.cert_nickname)
-- 
2.1.0

From 99949e6db6e44fe0c1245bd293a09dda899c9fce Mon Sep 17 00:00:00 2001
From: Martin Babinsky 
Date: Tue, 28 Apr 2015 16:34:33 +0200
Subject: [PATCH 2/2] rename httpd.service to ipa-httpd.service

The Apache server service controlled by HTTPInstance is renamed to
ipa-httpd.service.

https://fedorahosted.org/freeipa/ticket/4973
---
 install/share/Makefile.am   | 2 +-
 .../share/{httpd.service.template => ipa-httpd.service.template}| 0
 ipaplatform/redhat/services

Re: [Freeipa-devel] [PATCH 0046] Remove unneeded --ip-address option in ipa-adtrust-install

2015-04-28 Thread Gabe Alford 
On Tue, Apr 28, 2015 at 7:18 AM, Martin Basti  wrote:

>  On 27/04/15 15:02, Gabe Alford wrote:
>
>  Hello,
>
> Fix for https://fedorahosted.org/freeipa/ticket/4575
>
>  Thanks,
>
>  Gabe
>
>
>  Hello,
>
> thank you for your patch, looks good.
>
> However, IMO you can remove hostaddr variable as well, it was used for
> validate IP address.
> I could not find it used anywhere, did you?
>

Removed. It still seemed like it could be a logical check to have to me
which is why I kept it in the first place.
Updated patch attached.


> Martin^2
>
> --
> Martin Basti
>
>
From dae0bc832c5b8beadcba8d06cedc5bb61033c6d3 Mon Sep 17 00:00:00 2001
From: Gabe 
Date: Tue, 28 Apr 2015 07:34:07 -0600
Subject: [PATCH] Remove unneeded ip-address option in ipa-adtrust-install

https://fedorahosted.org/freeipa/ticket/4575
---
 install/tools/ipa-adtrust-install   | 36 +
 install/tools/man/ipa-adtrust-install.1 |  3 ---
 ipaserver/install/adtrustinstance.py|  4 +---
 3 files changed, 2 insertions(+), 41 deletions(-)

diff --git a/install/tools/ipa-adtrust-install b/install/tools/ipa-adtrust-install
index ac1d13a130f7ae295825dd1a16da2b3f946fe002..a4124078f712947b4575f9ceb078e4056425f319 100755
--- a/install/tools/ipa-adtrust-install
+++ b/install/tools/ipa-adtrust-install
@@ -39,8 +39,6 @@ def parse_options():
 parser = IPAOptionParser(version=version.VERSION)
 parser.add_option("-d", "--debug", dest="debug", action="store_true",
   default=False, help="print debugging information")
-parser.add_option("--ip-address", dest="ip_address",
-  type="ip", ip_local=True, help="Master Server IP Address")
 parser.add_option("--netbios-name", dest="netbios_name",
   help="NetBIOS name of the IPA domain")
 parser.add_option("--no-msdcs", dest="no_msdcs", action="store_true",
@@ -290,38 +288,6 @@ def main():
 if not options.unattended and not options.enable_compat:
 options.enable_compat = enable_compat_tree()
 
-# Check we have a public IP that is associated with the hostname
-ip = None
-try:
-hostaddr = resolve_host(api.env.host)
-if len(hostaddr) > 1:
-print >> sys.stderr, "The server hostname resolves to more than one address:"
-for addr in hostaddr:
-print >> sys.stderr, "  %s" % addr
-
-if options.ip_address:
-if str(options.ip_address) not in hostaddr:
-print >> sys.stderr, "Address passed in --ip-address did not match any resolved"
-print >> sys.stderr, "address!"
-sys.exit(1)
-print "Selected IP address:", str(options.ip_address)
-ip = options.ip_address
-else:
-if options.unattended:
-print >> sys.stderr, "Please use --ip-address option to specify the address"
-sys.exit(1)
-else:
-ip = read_ip_address(api.env.host, fstore)
-else:
-ip = hostaddr and ipautil.CheckedIPAddress(hostaddr[0], match_local=True)
-except Exception, e:
-print "Error: Invalid IP Address %s: %s" % (ip, e)
-print "Aborting installation"
-sys.exit(1)
-
-ip_address = str(ip)
-root_logger.debug("will use ip_address: %s\n", ip_address)
-
 admin_password = options.admin_password
 if not (options.unattended or admin_password):
 admin_password = read_admin_password(options.admin_name)
@@ -406,7 +372,7 @@ def main():
 smb = adtrustinstance.ADTRUSTInstance(fstore)
 smb.realm = api.env.realm
 smb.autobind = ipaldap.AUTOBIND_ENABLED
-smb.setup(api.env.host, ip_address, api.env.realm, api.env.domain,
+smb.setup(api.env.host, api.env.realm, api.env.domain,
   netbios_name, reset_netbios_name,
   options.rid_base, options.secondary_rid_base,
   options.no_msdcs, options.add_sids,
diff --git a/install/tools/man/ipa-adtrust-install.1 b/install/tools/man/ipa-adtrust-install.1
index b0aa8ceefc34698329b2a13d3adbcb204f08b3a9..a32eefb0e2dd4334b6dc3597b3643743ead56847 100644
--- a/install/tools/man/ipa-adtrust-install.1
+++ b/install/tools/man/ipa-adtrust-install.1
@@ -41,9 +41,6 @@ might be affected as well.
 \fB\-d\fR, \fB\-\-debug\fR
 Enable debug logging when more verbose output is needed
 .TP
-\fB\-\-ip\-address\fR=\fIIP_ADDRESS\fR
-The IP address of the IPA server. If not provided then this is determined based on the hostname of the server.
-.TP
 \fB\-\-netbios\-name\fR=\fINETBIOS_NAME\fR
 The NetBIOS name for the IPA domain. If not provided then this is determined
 based on the leading component of the DNS domain name. Running
diff --git a/ipaserver/install/adtrustinstance.py b/ipaserver/install/adtrustinstance.py
index b4d644fdbf784dd7936adc8eb085f4825cab797e..92c05f26a10c8f90bbe62ae9f6723d5e22ff3833 100644
--- a/ipaserver/install/ad

Re: [Freeipa-devel] [PATCH 0046] Remove unneeded --ip-address option in ipa-adtrust-install

2015-04-28 Thread Martin Basti 

On 27/04/15 15:02, Gabe Alford wrote:

Hello,

Fix for https://fedorahosted.org/freeipa/ticket/4575

Thanks,

Gabe



Hello,

thank you for your patch, looks good.

However, IMO you can remove hostaddr variable as well, it was used for 
validate IP address.

I could not find it used anywhere, did you?

Martin^2

--
Martin Basti

-- 
Manage your subscription for the Freeipa-devel mailing list:
https://www.redhat.com/mailman/listinfo/freeipa-devel
Contribute to FreeIPA: http://www.freeipa.org/page/Contribute/Code

Re: [Freeipa-devel] [PATCH 0029] suppress errors arising from deleting non-existent files during client uninstall

2015-04-28 Thread Martin Basti 

On 27/04/15 10:33, Martin Babinsky wrote:

On 04/24/2015 03:19 PM, Martin Basti wrote:

On 20/04/15 10:58, Martin Babinsky wrote:

On 04/20/2015 10:32 AM, Martin Basti wrote:

On 17/04/15 14:11, Martin Babinsky wrote:

On 04/17/2015 12:41 PM, Martin Babinsky wrote:

On 04/17/2015 12:36 PM, Martin Basti wrote:

On 17/04/15 12:33, Martin Babinsky wrote:

On 04/17/2015 12:04 PM, Martin Basti wrote:

On 15/04/15 15:53, Martin Babinsky wrote:

On 04/14/2015 04:24 PM, Martin Basti wrote:

On 14/04/15 16:12, Martin Basti wrote:

On 14/04/15 14:25, Martin Babinsky wrote:

This patch addresses
https://fedorahosted.org/freeipa/ticket/4966

The noise during rollback/uninstall is caused mainly by
unsuccessful
attempts to remove files that do not exist anymore. These
errors
are
now logged at debug level and do not pop-up to stdout/stderr.




Hello, thank you for the patch.

1)
The option add_warning is quite unclear to me. It does not 
show

warning but error. I suggest something like, show_hint,
show_user_action, or something show_additional_..., or
promt_manual_removal

Martin^2



Continue...

2)

 if file_exists(preferences_fname):
 try:
os.remove(preferences_fname)
 except OSError as e:
 log_file_removal_error(e, 
preferences_fname,

True)

In this case file not found error should never happen.

Could you remove the 'if file_exists' part and handle just
exception?


I just reverted this bit to original form in order to not fix
something that isn't broken. Is that ok?

3)
this is inconsistent with change above, choose one style 
please:


 if os.path.exists(ca_file):
 try:
 os.unlink(ca_file)
 except OSError, e:
 root_logger.error(
 "Failed to remove '%s': %s", 
ca_file, e)


--
Martin Basti



Attaching updated patch.


thanks,

just one nitpick, can you move the new function into
installutils, it
can be used in different scripts not just in ipaclient.



I'm not sure if it is a good idea as installutils is a part for
freeipa-server package.

Placing it there would create an unnecessary dependency of
freeipa-client on freeipa-server because of a single function.


you are right, I do not why I thought that ipa-client-install uses
installutils.

ACK


self-NACK, I will try to rewrite the patch in a slightly less dumb
way.

Sorry for the confusion.



Attaching updated patch which does the same but using a wrapper 
around

os.remove().

Jan suggested to keep the new function in 'ipa-client-install' and
move it around when we do installer re#$%@^ing.

Is that ok?


It looks better, ACK.


Jan NACKed your ACK.

Attaching updated patch.


Sorry, NACK

* Module ipa-client-install
ipa-client/ipa-install/ipa-client-install:791:
[E1121(too-many-function-args), uninstall] Too many positional arguments
for function call)
ipa-client/ipa-install/ipa-client-install:797:
[E1121(too-many-function-args), uninstall] Too many positional arguments
for function call)

consult with Honza if option which show prompt user to delete file
manually, should be there or not.



Updated patch attached.


ACK

--
Martin Basti

--
Manage your subscription for the Freeipa-devel mailing list:
https://www.redhat.com/mailman/listinfo/freeipa-devel
Contribute to FreeIPA: http://www.freeipa.org/page/Contribute/Code

Re: [Freeipa-devel] [PATCH 0014] emit a more helpful error messages when CA configuration fails

2015-04-28 Thread Martin Basti 

On 27/04/15 10:54, Martin Babinsky wrote:

On 04/24/2015 04:15 PM, Martin Basti wrote:

On 20/04/15 12:59, Martin Babinsky wrote:

On 04/17/2015 03:56 PM, Martin Babinsky wrote:

On 03/05/2015 01:11 PM, Martin Babinsky wrote:

https://fedorahosted.org/freeipa/ticket/4900



___
Freeipa-devel mailing list
Freeipa-devel@redhat.com
https://www.redhat.com/mailman/listinfo/freeipa-devel



Nobody to review this?



Attaching updated patches, one for ipa-4-1 (no DogtagInstance) and one
for master.




Hello, thank for patches:

1)
why is there

+PKI_UNINSTALL_LOG = paths.PKI_CA_UNINSTALL_LOG

I cannot find it used in patches?


Martin^2

--
Martin Basti


That was likely only my oversight. Attaching updated patches.


ACK

--
Martin Basti

--
Manage your subscription for the Freeipa-devel mailing list:
https://www.redhat.com/mailman/listinfo/freeipa-devel
Contribute to FreeIPA: http://www.freeipa.org/page/Contribute/Code

[Freeipa-devel] [PATCH 0321] Update and standardize copyright headers; introduce AUTHORS file

2015-04-28 Thread Petr Spacek 
Hello,

I'm going to add couple new files to the source tree and current copyright
header drove me mad, so here is (finally) a clenaup.


Update and standardize copyright headers; introduce AUTHORS file.

Dates in all headers were harmonized with Git history.
AUTHORS file lists all authors listed in Git history and source files too.

-- 
Petr^2 Spacek
From 06c00763378629977e458322bc4862719ce7f9a0 Mon Sep 17 00:00:00 2001
From: Petr Spacek 
Date: Tue, 28 Apr 2015 14:40:46 +0200
Subject: [PATCH] Update and standardize copyright headers; introduce AUTHORS
 file.

Dates in all headers were harmonized with Git history.
AUTHORS file lists all authors listed in Git history and source files too.
---
 AUTHORS  |  9 
 COPYING  | 26 +
 releng/bumpver.py|  3 +++
 releng/srcversion.py |  3 +++
 releng/trac.py   |  3 +++
 releng/tracvers.py   |  3 +++
 src/acl.c| 37 +-
 src/acl.h| 18 +--
 src/compat.h | 18 +--
 src/fs.c | 18 +--
 src/fs.h | 48 +--
 src/fwd_register.c   |  4 
 src/fwd_register.h   |  4 
 src/krb5_helper.c| 15 +---
 src/krb5_helper.h|  4 
 src/ldap_convert.c   | 18 +--
 src/ldap_convert.h   | 18 +--
 src/ldap_driver.c| 19 +---
 src/ldap_driver.h| 18 +--
 src/ldap_entry.c | 19 +---
 src/ldap_entry.h | 19 +---
 src/ldap_helper.c| 20 +---
 src/ldap_helper.h| 19 +---
 src/lock.c   | 18 +--
 src/lock.h   | 50 ++--
 src/log.c| 64 
 src/log.h| 18 +--
 src/rbt_helper.c |  4 
 src/rbt_helper.h |  4 
 src/rdlist.c | 19 +---
 src/rdlist.h | 19 +---
 src/semaphore.c  | 18 +--
 src/semaphore.h  | 18 +--
 src/settings.c   | 18 +--
 src/settings.h   | 18 +--
 src/str.c| 18 +--
 src/str.h| 18 +--
 src/syncrepl.c   | 18 +--
 src/syncrepl.h   | 18 +--
 src/types.h  | 18 +--
 src/util.h   | 18 +--
 src/zone_manager.c   | 18 +--
 src/zone_manager.h   | 19 +---
 src/zone_register.c  | 18 +--
 src/zone_register.h  | 18 +--
 45 files changed, 155 insertions(+), 657 deletions(-)
 create mode 100644 AUTHORS
 rewrite src/fs.h (76%)
 rewrite src/lock.h (74%)
 rewrite src/log.c (70%)

diff --git a/AUTHORS b/AUTHORS
new file mode 100644
index ..5a8014babcf5ee887f3db8f64f03621481701c11
--- /dev/null
+++ b/AUTHORS
@@ -0,0 +1,9 @@
+Adam Tkac 
+Jiri Kuncar 
+Lukas Slebodnik 
+Martin Nagy 
+Petr Spacek 
+Simo Sorce 
+Stephen Gallagher 
+Tomas Babej 
+Zoran Pericic 
diff --git a/COPYING b/COPYING
index d511905c1647a1e311e8b20d5930a37a9c2531cd..ca53a078ab8719a8be9d1e7177c4b9fd9683f5dc 100644
--- a/COPYING
+++ b/COPYING
@@ -1,3 +1,9 @@
+This program is free software; you can redistribute it and/or
+modify it under the terms of the GNU General Public License as
+published by the Free Software Foundation; version 2 or later
+
+---
+
 		GNU GENERAL PUBLIC LICENSE
 		   Version 2, June 1991
 
@@ -337,3 +343,23 @@ proprietary programs.  If your program is a subroutine library, you may
 consider it more useful to permit linking proprietary applications with the
 library.  If this is what you want to do, use the GNU Lesser General
 Public License instead of this License.
+
+---
+
+Portions of the code (snippets in acl.c and in ldap_helper.c)
+were taken from BIND 9 project:
+
+Copyright (C) 2004-2014  Internet Systems Consortium, Inc. ("ISC")
+Copyright (C) 2001-2003  Internet Software Consortium.
+
+Permission to use, copy, modify, and/or distribute this software for any
+purpose with or without fee is hereby granted, provided that the above
+copyright notice and this permission notice appear in all copies.
+
+THE SOFTWARE IS PROVIDED "AS IS" AND ISC DISCLAIMS ALL WARRANTIES WITH
+REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF MERCHANTABILITY
+AND FITNESS.  IN NO EVENT SHALL ISC BE LIABLE FOR ANY SPECIAL, DIRECT,
+INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES WHATSOEVER RESULTING FROM
+LOSS OF USE, DATA OR PROFITS, WHETHER IN AN ACTION OF CONTRACT, NEGLIGENCE
+OR OTHER TORTIOUS ACTION, ARISING OUT OF OR IN CONNECTION WITH THE USE OR
+PERFORMANCE OF THIS SOFTWARE.
diff --git a/releng/bumpver.py b/releng/bumpver.py
index 191ff2ffe68451f88ae112718a8

Re: [Freeipa-devel] [PATCH 0230] Server upgrade: fix comment in ldapupdater

2015-04-28 Thread Martin Basti 

On 27/04/15 18:42, David Kupka wrote:

On 04/16/2015 05:14 PM, Martin Basti wrote:

https://fedorahosted.org/freeipa/ticket/4904

Patch attached





I guess the rest of the comment is also outdated. Can you update it, too?


Updated patch attached.

--
Martin Basti

From d43c60e11eb61700e702bb968362d5fa23b5e782 Mon Sep 17 00:00:00 2001
From: Martin Basti 
Date: Thu, 16 Apr 2015 15:32:01 +0200
Subject: [PATCH] Server Upgrade: fix a comment in ldapupdater

DN sorting was removed in previous patches

https://fedorahosted.org/freeipa/ticket/4904
---
 ipaserver/install/ldapupdate.py | 7 +--
 1 file changed, 1 insertion(+), 6 deletions(-)

diff --git a/ipaserver/install/ldapupdate.py b/ipaserver/install/ldapupdate.py
index 1f30f4b215383a7964c578975b239ea4a57f68a8..171cb904f58ea415e12932dec8f38c02d9e1e688 100644
--- a/ipaserver/install/ldapupdate.py
+++ b/ipaserver/install/ldapupdate.py
@@ -769,12 +769,7 @@ class LDAPUpdate:
 
 def _delete_record(self, updates):
 """
-Run through all the updates again looking for any that should be
-deleted.
-
-This must use a reversed list so that the longest entries are
-considered first so we don't end up trying to delete a parent
-and child in the wrong order.
+Delete record
 """
 
 dn = updates['dn']
-- 
2.1.0

-- 
Manage your subscription for the Freeipa-devel mailing list:
https://www.redhat.com/mailman/listinfo/freeipa-devel
Contribute to FreeIPA: http://www.freeipa.org/page/Contribute/Code

[Freeipa-devel] [PATCHES 0235-0237] Use global DS write lock during upgrade

2015-04-28 Thread Martin Basti 

https://fedorahosted.org/freeipa/ticket/4925

I had to use ldif parser to edit DSE file instead (patch 236) and due to 
cyclic import caused by upgrade instace and dsintance I had to move 
realm_to_serverid from dsinstance to installutils.


Patches attached.

--
Martin Basti

From 4c89978091fdec7bd30a58fe26bef1070269231f Mon Sep 17 00:00:00 2001
From: Martin Basti 
Date: Mon, 27 Apr 2015 14:42:31 +0200
Subject: [PATCH 1/3] move realm_to_serverid to installutils module

To avoid cyclic imports realm_to_serverid function had to be moved to
installutils from dsinstance.

Required for: https://fedorahosted.org/freeipa/ticket/4925
---
 install/tools/ipa-ca-install|  2 +-
 install/tools/ipa-replica-install   |  3 ++-
 install/tools/ipa-server-install|  8 +---
 install/tools/ipa-upgradeconfig |  6 +++---
 install/tools/ipactl|  4 ++--
 ipaserver/install/adtrustinstance.py|  3 +--
 ipaserver/install/bindinstance.py   |  3 +--
 ipaserver/install/cainstance.py |  2 +-
 ipaserver/install/dsinstance.py |  9 +++--
 ipaserver/install/installutils.py   |  3 +++
 ipaserver/install/ipa_backup.py | 19 ---
 ipaserver/install/ipa_replica_prepare.py|  2 +-
 ipaserver/install/ipa_restore.py|  9 +
 ipaserver/install/ipa_server_certinstall.py |  2 +-
 ipaserver/install/krbinstance.py|  2 +-
 ipaserver/install/upgradeinstance.py|  3 +--
 16 files changed, 43 insertions(+), 37 deletions(-)

diff --git a/install/tools/ipa-ca-install b/install/tools/ipa-ca-install
index 6f8ff880b5d152197000f3c03f7e7b90f9945da0..f087d2d6a5138915008395cde4c461fc7602811b 100755
--- a/install/tools/ipa-ca-install
+++ b/install/tools/ipa-ca-install
@@ -284,7 +284,7 @@ def install_master(safe_options, options):
 check_ca()
 
 dirname = dsinstance.config_dirname(
-dsinstance.realm_to_serverid(realm_name))
+installutils.realm_to_serverid(realm_name))
 cadb = certs.CertDB(realm_name, subject_base=subject_base)
 dsdb = certs.CertDB(realm_name, nssdir=dirname, subject_base=subject_base)
 
diff --git a/install/tools/ipa-replica-install b/install/tools/ipa-replica-install
index 86931293fef247ea696eb1f821f287a03188da2d..b09a5f16e7cfe49337d88add5e20372f3a87031b 100755
--- a/install/tools/ipa-replica-install
+++ b/install/tools/ipa-replica-install
@@ -544,7 +544,8 @@ def main():
 fd.write("realm=%s\n" % config.realm_name)
 fd.write("domain=%s\n" % config.domain_name)
 fd.write("xmlrpc_uri=https://%s/ipa/xml\n"; % ipautil.format_netloc(config.host_name))
-fd.write("ldap_uri=ldapi://%%2fvar%%2frun%%2fslapd-%s.socket\n" % dsinstance.realm_to_serverid(config.realm_name))
+fd.write("ldap_uri=ldapi://%%2fvar%%2frun%%2fslapd-%s.socket\n" %
+ installutils.realm_to_serverid(config.realm_name))
 if ipautil.file_exists(config.dir + "/cacert.p12"):
 fd.write("enable_ra=True\n")
 fd.write("ra_plugin=dogtag\n")
diff --git a/install/tools/ipa-server-install b/install/tools/ipa-server-install
index 56a43770d95387762bce09634bd1056ba7f20576..1fe5bd9ea6294a79728ed68216ea407857790d34 100755
--- a/install/tools/ipa-server-install
+++ b/install/tools/ipa-server-install
@@ -655,7 +655,8 @@ def uninstall():
   'mean your system hasn\'t be restored to its pre-installation state.' % SYSRESTORE_DIR_PATH)
 
 # Note that this name will be wrong after the first uninstall.
-dirname = dsinstance.config_dirname(dsinstance.realm_to_serverid(api.env.realm))
+dirname = dsinstance.config_dirname(
+installutils.realm_to_serverid(api.env.realm))
 dirs = [dirname, dogtag_constants.ALIAS_DIR, certs.NSS_DIR]
 ids = certmonger.check_state(dirs)
 if ids:
@@ -666,7 +667,7 @@ def uninstall():
 
 def set_subject_in_config(realm_name, dm_password, suffix, subject_base):
 ldapuri = 'ldapi://%%2fvar%%2frun%%2fslapd-%s.socket' % (
-dsinstance.realm_to_serverid(realm_name)
+installutils.realm_to_serverid(realm_name)
 )
 try:
 conn = ldap2(shared_instance=False, ldap_uri=ldapuri, base_dn=suffix)
@@ -1134,7 +1135,8 @@ def main():
 fd.write("realm=%s\n" % realm_name)
 fd.write("domain=%s\n" % domain_name)
 fd.write("xmlrpc_uri=https://%s/ipa/xml\n"; % format_netloc(host_name))
-fd.write("ldap_uri=ldapi://%%2fvar%%2frun%%2fslapd-%s.socket\n" % dsinstance.realm_to_serverid(realm_name))
+fd.write("ldap_uri=ldapi://%%2fvar%%2frun%%2fslapd-%s.socket\n" %
+ installutils.realm_to_serverid(realm_name))
 if setup_ca:
 fd.write("enable_ra=True\n")
 fd.write("ra_plugin=dogtag\n")
diff --git a/install/tools/ipa-upgradeconfig b/install/tools/ipa-upgradeconfig
index 8159ce244b873962df36409b884bc3ee70621f2f..dfef1e0aa8b1507b7aa4907e9b688ce99253b87c 1

Re: [Freeipa-devel] [PATCH 0001] Fixed incorrect ldap_uri population

2015-04-28 Thread Jan Cholasta 

Hi,

Dne 24.4.2015 v 16:27 Oleg Fayans napsal(a):

This one is even more correct.

On 04/24/2015 04:05 PM, Oleg Fayans wrote:

Corresponding ticket is
https://fedorahosted.org/freeipa/ticket/5002


thanks for the patch, but the bug is purely hypothetical, as the 
__ldap_uri attribute is always set.


Anyway, you shouldn't use "'name' in dir(obj)" to check if attribute is 
set, there is "hasattr(obj, 'name')" for that.


Also, "__ldap_uri" starts with two underscores, so the name is mangled 
to "_ldap2__ldap_uri", which means the check in your patch will always 
fail and ldap_uri will always return self.api.env.ldap_uri, which is in 
fact more wrong than the current behavior.


Honza

--
Jan Cholasta

--
Manage your subscription for the Freeipa-devel mailing list:
https://www.redhat.com/mailman/listinfo/freeipa-devel
Contribute to FreeIPA: http://www.freeipa.org/page/Contribute/Code

Re: [Freeipa-devel] [PATCH] 0005 User life cycle: del/mod/find/show stageuser commands

2015-04-28 Thread David Kupka 

On 04/28/2015 10:28 AM, thierry bordaz wrote:

On 04/28/2015 10:23 AM, David Kupka wrote:

On 04/16/2015 01:00 PM, thierry bordaz wrote:

Hello,

Here is the next patch for User life cycle that introduces
del/mod/find and show stageuser plugin commands.

  * -User Life Cycle (create containers and scoping  DS plugins):
*pushed*
  * 0001-User-Life-Cycle-Exclude-subtree-for-ipaUniqueID-gene.patch:
*pushed*
  * 0002-User-life-cycle-stageuser-add-verb.patch: *pushed*
  * 0007-User-life-cycle-allows-MODRDN-from-ldap2.patch: *pushed*
  * 0003-User-life-cycle-new-stageuser-commands-del-mod-find-*under
review *(this one)**
  * 0004-User-life-cycle-new-stageuser-commands-activate.patch
  * 0005-User-life-cycle-new-stageuser-commands-activate-prov.patch
  * 0006-User-life-cycle-user-del-supports-permanently-preser.patch
  * 0008-User-life-cycle-user-find-support-finding-delete-use.patch
  * 0009-User-life-cycle-support-of-user-undel.patch
  * 0010-User-life-cycle-DNA-DS-plugin-should-exclude-provisi.patch
  * 0011-User-life-cycle-lockout-provisioning-stage-and-delet.patch
  * 0012-User-life-cycle-Create-stage-Admin-provisioning-acco.patch
  * 0013-User-life-cycle-Stage-Admin-permission-priviledge.patch

Thanks
thierry





Hi Thierry,
thanks for the patch, the code looks good to me but there is probably
a bug in ACIs.
After creating a stage user and setting password for him I can kinit
as the stage user. I'm unable to login to the IPA client and id
command for this stage user responds "no such user" but I can kinit
and invoke ipa commands.

Steps:
0. build freeipa with your patch
1. # ipa-server-install
2. $ kinit admin
3. $ ipa stageuser-add suser0 --first Stage --last User --password
4. $ kdestroy
5. $ kinit suser0
6. $ ipa user-find

Actual:
Prints out list of ipa users.

Expected:
kinit fails with "suser0@... not found in Kerberos database"


Hi David,

Thank you so much for having looked at this patch :-)
You are right. The Staging users (as well as the Delete users) are not
lockout in that patch.
The patch
0011-User-life-cycle-lockout-provisioning-stage-and-delet.patch will
take care of this.

Do you prefer that I merged the two patches right now ?

thanks
thierry



Hi Thierry,
no, it is not necessary to merge the patches it's ok to have it 
separated. I'm not sure if the patch should be pushed now or rather wait 
and push it together with the others.

I'm looking forward to next ULC patches from you.

--
David Kupka

--
Manage your subscription for the Freeipa-devel mailing list:
https://www.redhat.com/mailman/listinfo/freeipa-devel
Contribute to FreeIPA: http://www.freeipa.org/page/Contribute/Code

Re: [Freeipa-devel] [PATCH] 0005 User life cycle: del/mod/find/show stageuser commands

2015-04-28 Thread thierry bordaz 

On 04/28/2015 10:23 AM, David Kupka wrote:

On 04/16/2015 01:00 PM, thierry bordaz wrote:

Hello,

Here is the next patch for User life cycle that introduces
del/mod/find and show stageuser plugin commands.

  * -User Life Cycle (create containers and scoping  DS plugins):
*pushed*
  * 0001-User-Life-Cycle-Exclude-subtree-for-ipaUniqueID-gene.patch:
*pushed*
  * 0002-User-life-cycle-stageuser-add-verb.patch: *pushed*
  * 0007-User-life-cycle-allows-MODRDN-from-ldap2.patch: *pushed*
  * 0003-User-life-cycle-new-stageuser-commands-del-mod-find-*under
review *(this one)**
  * 0004-User-life-cycle-new-stageuser-commands-activate.patch
  * 0005-User-life-cycle-new-stageuser-commands-activate-prov.patch
  * 0006-User-life-cycle-user-del-supports-permanently-preser.patch
  * 0008-User-life-cycle-user-find-support-finding-delete-use.patch
  * 0009-User-life-cycle-support-of-user-undel.patch
  * 0010-User-life-cycle-DNA-DS-plugin-should-exclude-provisi.patch
  * 0011-User-life-cycle-lockout-provisioning-stage-and-delet.patch
  * 0012-User-life-cycle-Create-stage-Admin-provisioning-acco.patch
  * 0013-User-life-cycle-Stage-Admin-permission-priviledge.patch

Thanks
thierry





Hi Thierry,
thanks for the patch, the code looks good to me but there is probably 
a bug in ACIs.
After creating a stage user and setting password for him I can kinit 
as the stage user. I'm unable to login to the IPA client and id 
command for this stage user responds "no such user" but I can kinit 
and invoke ipa commands.


Steps:
0. build freeipa with your patch
1. # ipa-server-install
2. $ kinit admin
3. $ ipa stageuser-add suser0 --first Stage --last User --password
4. $ kdestroy
5. $ kinit suser0
6. $ ipa user-find

Actual:
Prints out list of ipa users.

Expected:
kinit fails with "suser0@... not found in Kerberos database"


Hi David,

Thank you so much for having looked at this patch :-)
You are right. The Staging users (as well as the Delete users) are not 
lockout in that patch.
The patch 
0011-User-life-cycle-lockout-provisioning-stage-and-delet.patch will 
take care of this.


Do you prefer that I merged the two patches right now ?

thanks
thierry
-- 
Manage your subscription for the Freeipa-devel mailing list:
https://www.redhat.com/mailman/listinfo/freeipa-devel
Contribute to FreeIPA: http://www.freeipa.org/page/Contribute/Code

Re: [Freeipa-devel] [PATCH] 0005 User life cycle: del/mod/find/show stageuser commands

2015-04-28 Thread David Kupka 

On 04/16/2015 01:00 PM, thierry bordaz wrote:

Hello,

Here is the next patch for User life cycle that introduces
del/mod/find and show stageuser plugin commands.

  * -User Life Cycle (create containers and scoping  DS plugins):
*pushed*
  * 0001-User-Life-Cycle-Exclude-subtree-for-ipaUniqueID-gene.patch:
*pushed*
  * 0002-User-life-cycle-stageuser-add-verb.patch: *pushed*
  * 0007-User-life-cycle-allows-MODRDN-from-ldap2.patch: *pushed*
  * 0003-User-life-cycle-new-stageuser-commands-del-mod-find-*under
review *(this one)**
  * 0004-User-life-cycle-new-stageuser-commands-activate.patch
  * 0005-User-life-cycle-new-stageuser-commands-activate-prov.patch
  * 0006-User-life-cycle-user-del-supports-permanently-preser.patch
  * 0008-User-life-cycle-user-find-support-finding-delete-use.patch
  * 0009-User-life-cycle-support-of-user-undel.patch
  * 0010-User-life-cycle-DNA-DS-plugin-should-exclude-provisi.patch
  * 0011-User-life-cycle-lockout-provisioning-stage-and-delet.patch
  * 0012-User-life-cycle-Create-stage-Admin-provisioning-acco.patch
  * 0013-User-life-cycle-Stage-Admin-permission-priviledge.patch

Thanks
thierry





Hi Thierry,
thanks for the patch, the code looks good to me but there is probably a 
bug in ACIs.
After creating a stage user and setting password for him I can kinit as 
the stage user. I'm unable to login to the IPA client and id command for 
this stage user responds "no such user" but I can kinit and invoke ipa 
commands.


Steps:
0. build freeipa with your patch
1. # ipa-server-install
2. $ kinit admin
3. $ ipa stageuser-add suser0 --first Stage --last User --password
4. $ kdestroy
5. $ kinit suser0
6. $ ipa user-find

Actual:
Prints out list of ipa users.

Expected:
kinit fails with "suser0@... not found in Kerberos database"

--
David Kupka

--
Manage your subscription for the Freeipa-devel mailing list:
https://www.redhat.com/mailman/listinfo/freeipa-devel
Contribute to FreeIPA: http://www.freeipa.org/page/Contribute/Code