Re: [Gluster-users] "Another Transaction is in progres..."

[Krist van Besien]

Thanks for the suggestion, this solved it for us, and we probably found the
cause as well. We had performance co-pilot running and it was continously
enabling profiling on volumes...
We found the reference to the node that had the lock, and restarted
glusterd on that node, and all went well from there on.

Krist


On 31 May 2017 at 15:56, Vijay Bellur  wrote:

>
>
> On Wed, May 31, 2017 at 9:32 AM, Krist van Besien 
> wrote:
>
>> Hi all,
>>
>> I am trying to do trivial things, like setting quota, or just querying
>> the status and keep getting
>>
>> "Another transaction is in progres for "
>>
>> These messages pop up, then disappear for a while, then pop up again...
>>
>> What do these messages mean? How do I figure out which "transaction" is
>> meant here, and what do I do about it?
>>
>
>
> This message usually means that a different gluster command is being
> executed in the cluster. Most gluster commands are serialized by a cluster
> wide lock. Upon not being able to acquire the cluster lock, this message is
> displayed.
>
> You can check /var/log/glusterfs/cmd_history.log on all storage nodes to
> observe what other commands are in progress at the time of getting this
> error message. Are you per chance using oVirt to manage Gluster? oVirt
> periodically does a "gluster volume status" to determine the volume health
> and that can conflict with other commands being executed.
>
> Regards,
> Vijay
>
>


-- 
Vriendelijke Groet |  Best Regards | Freundliche Grüße | Cordialement
--
Krist van Besien | Senior Architect | Red Hat EMEA Cloud Practice | RHCE |
RHCSA Open Stack
@: kr...@redhat.com | M: +41-79-5936260
___
Gluster-users mailing list
Gluster-users@gluster.org
http://lists.gluster.org/mailman/listinfo/gluster-users

Re: [Gluster-users] "Another Transaction is in progres..."

[Atin Mukherjee]

On Wed, May 31, 2017 at 7:31 PM, Erekle Magradze <
erekle.magra...@recogizer.de> wrote:

> Hello,
> I think I have the same issues, I am using gluster as the VM image storage
> under oVirt, would it make sense to restart gluster services on all hosts
> (of course with turned off VMs)
>

If two transactions collide then one of the transaction will fail, but the
theory is you shouldn't be ending up with a stale lock. If further commands
continue to fail then that indicates glusterd has ended up with a stale
lock and then a glusterd service restart is required. We have seen issues
when any op-sm originated transaction (gluster v rebalance, gluster v
profile) collides with any mgmt v3 or syncop based transactions (other
gluster commands) where due to race we can end up with a stale lock but
that will be a rare scenario.




> Cheers
> Erekle
>
>
> On 05/31/2017 03:56 PM, Vijay Bellur wrote:
>
>
>
> On Wed, May 31, 2017 at 9:32 AM, Krist van Besien 
> wrote:
>
>> Hi all,
>>
>> I am trying to do trivial things, like setting quota, or just querying
>> the status and keep getting
>>
>> "Another transaction is in progres for "
>>
>> These messages pop up, then disappear for a while, then pop up again...
>>
>> What do these messages mean? How do I figure out which "transaction" is
>> meant here, and what do I do about it?
>>
>
>
> This message usually means that a different gluster command is being
> executed in the cluster. Most gluster commands are serialized by a cluster
> wide lock. Upon not being able to acquire the cluster lock, this message is
> displayed.
>
> You can check /var/log/glusterfs/cmd_history.log on all storage nodes to
> observe what other commands are in progress at the time of getting this
> error message. Are you per chance using oVirt to manage Gluster? oVirt
> periodically does a "gluster volume status" to determine the volume health
> and that can conflict with other commands being executed.
>
> Regards,
> Vijay
>
>
>
> ___
> Gluster-users mailing 
> listGluster-users@gluster.orghttp://lists.gluster.org/mailman/listinfo/gluster-users
>
>
> --
> Recogizer Group GmbH
>
> Erekle Magradze
> Lead Big Data Engineering & DevOps
> Rheinwerkallee 2, 53227 Bonn
> Tel: +49 228 29974555
>
> E-Mail erekle.magra...@recogizer.de
> Web: www.recogizer.com
>
> Recogizer auf LinkedIn https://www.linkedin.com/company-beta/10039182/
> Folgen Sie uns auf Twitter https://twitter.com/recogizer
>
> -
> Recogizer Group GmbH
> Geschäftsführer: Oliver Habisch, Carsten Kreutze
> Handelsregister: Amtsgericht Bonn HRB 20724
> Sitz der Gesellschaft: Bonn; USt-ID-Nr.: DE294195993
>
> Diese E-Mail enthält vertrauliche und/oder rechtlich geschützte Informationen.
> Wenn Sie nicht der richtige Adressat sind oder diese E-Mail irrtümlich 
> erhalten haben,
> informieren Sie bitte sofort den Absender und löschen Sie diese Mail.
> Das unerlaubte Kopieren sowie die unbefugte Weitergabe dieser Mail und der 
> darin enthaltenen Informationen ist nicht gestattet.
>
>
> ___
> Gluster-users mailing list
> Gluster-users@gluster.org
> http://lists.gluster.org/mailman/listinfo/gluster-users
>
___
Gluster-users mailing list
Gluster-users@gluster.org
http://lists.gluster.org/mailman/listinfo/gluster-users

Re: [Gluster-users] "Another Transaction is in progres..."

[Erekle Magradze]

Hello,
I think I have the same issues, I am using gluster as the VM image 
storage under oVirt, would it make sense to restart gluster services on 
all hosts (of course with turned off VMs)

Cheers
Erekle

On 05/31/2017 03:56 PM, Vijay Bellur wrote:



On Wed, May 31, 2017 at 9:32 AM, Krist van Besien > wrote:


Hi all,

I am trying to do trivial things, like setting quota, or just
querying the status and keep getting

"Another transaction is in progres for "

These messages pop up, then disappear for a while, then pop up
again...

What do these messages mean? How do I figure out which
"transaction" is meant here, and what do I do about it?



This message usually means that a different gluster command is being 
executed in the cluster. Most gluster commands are serialized by a 
cluster wide lock. Upon not being able to acquire the cluster lock, 
this message is displayed.


You can check /var/log/glusterfs/cmd_history.log on all storage nodes 
to observe what other commands are in progress at the time of getting 
this error message. Are you per chance using oVirt to manage Gluster? 
oVirt periodically does a "gluster volume status" to determine the 
volume health and that can conflict with other commands being executed.


Regards,
Vijay



___
Gluster-users mailing list
Gluster-users@gluster.org
http://lists.gluster.org/mailman/listinfo/gluster-users


--
Recogizer Group GmbH

Erekle Magradze
Lead Big Data Engineering & DevOps
Rheinwerkallee 2, 53227 Bonn
Tel: +49 228 29974555

E-Mail erekle.magra...@recogizer.de
Web: www.recogizer.com
 
Recogizer auf LinkedIn https://www.linkedin.com/company-beta/10039182/

Folgen Sie uns auf Twitter https://twitter.com/recogizer
 
-

Recogizer Group GmbH
Geschäftsführer: Oliver Habisch, Carsten Kreutze
Handelsregister: Amtsgericht Bonn HRB 20724
Sitz der Gesellschaft: Bonn; USt-ID-Nr.: DE294195993
 
Diese E-Mail enthält vertrauliche und/oder rechtlich geschützte Informationen.

Wenn Sie nicht der richtige Adressat sind oder diese E-Mail irrtümlich erhalten 
haben,
informieren Sie bitte sofort den Absender und löschen Sie diese Mail.
Das unerlaubte Kopieren sowie die unbefugte Weitergabe dieser Mail und der 
darin enthaltenen Informationen ist nicht gestattet.

___
Gluster-users mailing list
Gluster-users@gluster.org
http://lists.gluster.org/mailman/listinfo/gluster-users

Re: [Gluster-users] "Another Transaction is in progres..."

[Vijay Bellur]

On Wed, May 31, 2017 at 9:32 AM, Krist van Besien  wrote:

> Hi all,
>
> I am trying to do trivial things, like setting quota, or just querying the
> status and keep getting
>
> "Another transaction is in progres for "
>
> These messages pop up, then disappear for a while, then pop up again...
>
> What do these messages mean? How do I figure out which "transaction" is
> meant here, and what do I do about it?
>


This message usually means that a different gluster command is being
executed in the cluster. Most gluster commands are serialized by a cluster
wide lock. Upon not being able to acquire the cluster lock, this message is
displayed.

You can check /var/log/glusterfs/cmd_history.log on all storage nodes to
observe what other commands are in progress at the time of getting this
error message. Are you per chance using oVirt to manage Gluster? oVirt
periodically does a "gluster volume status" to determine the volume health
and that can conflict with other commands being executed.

Regards,
Vijay
___
Gluster-users mailing list
Gluster-users@gluster.org
http://lists.gluster.org/mailman/listinfo/gluster-users