Change in vdsm[master]: remoteFileHandler: config logging in spawned process
Ayal Baron has posted comments on this change. Change subject: remoteFileHandler: config logging in spawned process .. Patch Set 7: (1 comment) File vdsm/storage/remoteFileHandler.py Line 46: import misc Line 47: import fileUtils Line 48: import zombieReaper Line 49: from vdsm import constants Line 50: I'm not sure the complexity of it is worth the trouble considering http://gerrit.ovirt.org/14626 has been merged. Can you show why this is needed after this patch? Line 51: Line 52: # Crabs are known for their remote process calls Line 53: LENGTH_STRUCT_FMT = Q Line 54: LENGTH_STRUCT_LENGTH = calcsize(LENGTH_STRUCT_FMT) -- To view, visit http://gerrit.ovirt.org/9182 To unsubscribe, visit http://gerrit.ovirt.org/settings Gerrit-MessageType: comment Gerrit-Change-Id: I1b3b9e2837d632c7532fcd8f7306ed50b0865b5c Gerrit-PatchSet: 7 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Zhou Zheng Sheng zhshz...@linux.vnet.ibm.com Gerrit-Reviewer: Allon Mureinik amure...@redhat.com Gerrit-Reviewer: Ayal Baron aba...@redhat.com Gerrit-Reviewer: Better Saggi bettersa...@gmail.com Gerrit-Reviewer: Dan Kenigsberg dan...@redhat.com Gerrit-Reviewer: Federico Simoncelli fsimo...@redhat.com Gerrit-Reviewer: Mark Wu wu...@linux.vnet.ibm.com Gerrit-Reviewer: Saggi Mizrahi smizr...@redhat.com Gerrit-Reviewer: ShaoHe Feng shao...@linux.vnet.ibm.com Gerrit-Reviewer: Shu Ming shum...@linux.vnet.ibm.com Gerrit-Reviewer: Zhou Zheng Sheng zhshz...@linux.vnet.ibm.com Gerrit-Reviewer: oVirt Jenkins CI Server Gerrit-HasComments: Yes ___ vdsm-patches mailing list vdsm-patches@lists.fedorahosted.org https://lists.fedorahosted.org/mailman/listinfo/vdsm-patches
Change in vdsm[master]: remoteFileHandler: config logging in spawned process
Zhou Zheng Sheng has abandoned this change. Change subject: remoteFileHandler: config logging in spawned process .. Abandoned http://gerrit.ovirt.org/14626 covers this case and it's simpler. -- To view, visit http://gerrit.ovirt.org/9182 To unsubscribe, visit http://gerrit.ovirt.org/settings Gerrit-MessageType: abandon Gerrit-Change-Id: I1b3b9e2837d632c7532fcd8f7306ed50b0865b5c Gerrit-PatchSet: 7 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Zhou Zheng Sheng zhshz...@linux.vnet.ibm.com Gerrit-Reviewer: Allon Mureinik amure...@redhat.com Gerrit-Reviewer: Ayal Baron aba...@redhat.com Gerrit-Reviewer: Better Saggi bettersa...@gmail.com Gerrit-Reviewer: Dan Kenigsberg dan...@redhat.com Gerrit-Reviewer: Federico Simoncelli fsimo...@redhat.com Gerrit-Reviewer: Mark Wu wu...@linux.vnet.ibm.com Gerrit-Reviewer: Saggi Mizrahi smizr...@redhat.com Gerrit-Reviewer: ShaoHe Feng shao...@linux.vnet.ibm.com Gerrit-Reviewer: Shu Ming shum...@linux.vnet.ibm.com Gerrit-Reviewer: Zhou Zheng Sheng zhshz...@linux.vnet.ibm.com Gerrit-Reviewer: oVirt Jenkins CI Server ___ vdsm-patches mailing list vdsm-patches@lists.fedorahosted.org https://lists.fedorahosted.org/mailman/listinfo/vdsm-patches
Change in vdsm[master]: remoteFileHandler: config logging in spawned process
Dan Kenigsberg has posted comments on this change. Change subject: remoteFileHandler: config logging in spawned process .. Patch Set 7: (1 comment) File vdsm/storage/remoteFileHandler.py Line 46: import misc Line 47: import fileUtils Line 48: import zombieReaper Line 49: from vdsm import constants Line 50: I am so sorry that I did not notice your response and question. Yes, using syslog to queue the log messages is acceptable - if you can do that by dropping a file into /etc/rsyslog.d. I wouldn't mind if you queue vdsm-propper log messages so that we have one linear /var/log/vdsm/vdsm.log with everything inside. Line 51: Line 52: # Crabs are known for their remote process calls Line 53: LENGTH_STRUCT_FMT = Q Line 54: LENGTH_STRUCT_LENGTH = calcsize(LENGTH_STRUCT_FMT) -- To view, visit http://gerrit.ovirt.org/9182 To unsubscribe, visit http://gerrit.ovirt.org/settings Gerrit-MessageType: comment Gerrit-Change-Id: I1b3b9e2837d632c7532fcd8f7306ed50b0865b5c Gerrit-PatchSet: 7 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Zhou Zheng Sheng zhshz...@linux.vnet.ibm.com Gerrit-Reviewer: Allon Mureinik amure...@redhat.com Gerrit-Reviewer: Dan Kenigsberg dan...@redhat.com Gerrit-Reviewer: Federico Simoncelli fsimo...@redhat.com Gerrit-Reviewer: Mark Wu wu...@linux.vnet.ibm.com Gerrit-Reviewer: Saggi Mizrahi smizr...@redhat.com Gerrit-Reviewer: ShaoHe Feng shao...@linux.vnet.ibm.com Gerrit-Reviewer: Shu Ming shum...@linux.vnet.ibm.com Gerrit-Reviewer: Zhou Zheng Sheng zhshz...@linux.vnet.ibm.com Gerrit-Reviewer: oVirt Jenkins CI Server Gerrit-HasComments: Yes ___ vdsm-patches mailing list vdsm-patches@lists.fedorahosted.org https://lists.fedorahosted.org/mailman/listinfo/vdsm-patches
Change in vdsm[master]: remoteFileHandler: config logging in spawned process
oVirt Jenkins CI Server has posted comments on this change. Change subject: remoteFileHandler: config logging in spawned process .. Patch Set 7: Build Started http://jenkins.ovirt.org/job/vdsm_pep8_gerrit/2053/ (2/2) -- To view, visit http://gerrit.ovirt.org/9182 To unsubscribe, visit http://gerrit.ovirt.org/settings Gerrit-MessageType: comment Gerrit-Change-Id: I1b3b9e2837d632c7532fcd8f7306ed50b0865b5c Gerrit-PatchSet: 7 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Zhou Zheng Sheng zhshz...@linux.vnet.ibm.com Gerrit-Reviewer: Allon Mureinik amure...@redhat.com Gerrit-Reviewer: Dan Kenigsberg dan...@redhat.com Gerrit-Reviewer: Federico Simoncelli fsimo...@redhat.com Gerrit-Reviewer: Mark Wu wu...@linux.vnet.ibm.com Gerrit-Reviewer: Saggi Mizrahi smizr...@redhat.com Gerrit-Reviewer: ShaoHe Feng shao...@linux.vnet.ibm.com Gerrit-Reviewer: Shu Ming shum...@linux.vnet.ibm.com Gerrit-Reviewer: Zhou Zheng Sheng zhshz...@linux.vnet.ibm.com Gerrit-Reviewer: oVirt Jenkins CI Server ___ vdsm-patches mailing list vdsm-patches@lists.fedorahosted.org https://lists.fedorahosted.org/mailman/listinfo/vdsm-patches
Change in vdsm[master]: remoteFileHandler: config logging in spawned process
oVirt Jenkins CI Server has posted comments on this change. Change subject: remoteFileHandler: config logging in spawned process .. Patch Set 7: Build Successful http://jenkins.ovirt.org/job/vdsm_pep8_gerrit/2053/ : SUCCESS http://jenkins.ovirt.org/job/vdsm_unit_tests_gerrit/2111/ : SUCCESS -- To view, visit http://gerrit.ovirt.org/9182 To unsubscribe, visit http://gerrit.ovirt.org/settings Gerrit-MessageType: comment Gerrit-Change-Id: I1b3b9e2837d632c7532fcd8f7306ed50b0865b5c Gerrit-PatchSet: 7 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Zhou Zheng Sheng zhshz...@linux.vnet.ibm.com Gerrit-Reviewer: Allon Mureinik amure...@redhat.com Gerrit-Reviewer: Dan Kenigsberg dan...@redhat.com Gerrit-Reviewer: Federico Simoncelli fsimo...@redhat.com Gerrit-Reviewer: Mark Wu wu...@linux.vnet.ibm.com Gerrit-Reviewer: Saggi Mizrahi smizr...@redhat.com Gerrit-Reviewer: ShaoHe Feng shao...@linux.vnet.ibm.com Gerrit-Reviewer: Shu Ming shum...@linux.vnet.ibm.com Gerrit-Reviewer: Zhou Zheng Sheng zhshz...@linux.vnet.ibm.com Gerrit-Reviewer: oVirt Jenkins CI Server ___ vdsm-patches mailing list vdsm-patches@lists.fedorahosted.org https://lists.fedorahosted.org/mailman/listinfo/vdsm-patches
Change in vdsm[master]: remoteFileHandler: config logging in spawned process
Zhou Zheng Sheng has posted comments on this change. Change subject: remoteFileHandler: config logging in spawned process .. Patch Set 7: Thanks Shao He, Ming, Dan, Saggi, Allon, Tal, Mark. I rebase the patch and test it. The new patch set configures logging after closeFDs(), otherwise the opened log file handle will be closed as well. -- To view, visit http://gerrit.ovirt.org/9182 To unsubscribe, visit http://gerrit.ovirt.org/settings Gerrit-MessageType: comment Gerrit-Change-Id: I1b3b9e2837d632c7532fcd8f7306ed50b0865b5c Gerrit-PatchSet: 7 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Zhou Zheng Sheng zhshz...@linux.vnet.ibm.com Gerrit-Reviewer: Allon Mureinik amure...@redhat.com Gerrit-Reviewer: Dan Kenigsberg dan...@redhat.com Gerrit-Reviewer: Federico Simoncelli fsimo...@redhat.com Gerrit-Reviewer: Mark Wu wu...@linux.vnet.ibm.com Gerrit-Reviewer: Saggi Mizrahi smizr...@redhat.com Gerrit-Reviewer: ShaoHe Feng shao...@linux.vnet.ibm.com Gerrit-Reviewer: Shu Ming shum...@linux.vnet.ibm.com Gerrit-Reviewer: Zhou Zheng Sheng zhshz...@linux.vnet.ibm.com Gerrit-Reviewer: oVirt Jenkins CI Server ___ vdsm-patches mailing list vdsm-patches@lists.fedorahosted.org https://lists.fedorahosted.org/mailman/listinfo/vdsm-patches
Change in vdsm[master]: remoteFileHandler: config logging in spawned process
Allon Mureinik has posted comments on this change. Change subject: remoteFileHandler: config logging in spawned process .. Patch Set 7: Looks good to me, but someone else must approve -- To view, visit http://gerrit.ovirt.org/9182 To unsubscribe, visit http://gerrit.ovirt.org/settings Gerrit-MessageType: comment Gerrit-Change-Id: I1b3b9e2837d632c7532fcd8f7306ed50b0865b5c Gerrit-PatchSet: 7 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Zhou Zheng Sheng zhshz...@linux.vnet.ibm.com Gerrit-Reviewer: Allon Mureinik amure...@redhat.com Gerrit-Reviewer: Dan Kenigsberg dan...@redhat.com Gerrit-Reviewer: Federico Simoncelli fsimo...@redhat.com Gerrit-Reviewer: Mark Wu wu...@linux.vnet.ibm.com Gerrit-Reviewer: Saggi Mizrahi smizr...@redhat.com Gerrit-Reviewer: ShaoHe Feng shao...@linux.vnet.ibm.com Gerrit-Reviewer: Shu Ming shum...@linux.vnet.ibm.com Gerrit-Reviewer: Zhou Zheng Sheng zhshz...@linux.vnet.ibm.com Gerrit-Reviewer: oVirt Jenkins CI Server ___ vdsm-patches mailing list vdsm-patches@lists.fedorahosted.org https://lists.fedorahosted.org/mailman/listinfo/vdsm-patches
Change in vdsm[master]: remoteFileHandler: config logging in spawned process
Dan Kenigsberg has posted comments on this change. Change subject: remoteFileHandler: config logging in spawned process .. Patch Set 7: I would prefer that you didn't submit this (1 inline comment) File vdsm/storage/remoteFileHandler.py Line 46: import misc Line 47: import fileUtils Line 48: import zombieReaper Line 49: from vdsm import constants Line 50: What is the members penalty of this? And worse - wouldn't it be awfully raceful if 100 processes write to the same log file? Line 51: Line 52: # Crabs are known for their remote process calls Line 53: LENGTH_STRUCT_FMT = Q Line 54: LENGTH_STRUCT_LENGTH = calcsize(LENGTH_STRUCT_FMT) -- To view, visit http://gerrit.ovirt.org/9182 To unsubscribe, visit http://gerrit.ovirt.org/settings Gerrit-MessageType: comment Gerrit-Change-Id: I1b3b9e2837d632c7532fcd8f7306ed50b0865b5c Gerrit-PatchSet: 7 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Zhou Zheng Sheng zhshz...@linux.vnet.ibm.com Gerrit-Reviewer: Allon Mureinik amure...@redhat.com Gerrit-Reviewer: Dan Kenigsberg dan...@redhat.com Gerrit-Reviewer: Federico Simoncelli fsimo...@redhat.com Gerrit-Reviewer: Mark Wu wu...@linux.vnet.ibm.com Gerrit-Reviewer: Saggi Mizrahi smizr...@redhat.com Gerrit-Reviewer: ShaoHe Feng shao...@linux.vnet.ibm.com Gerrit-Reviewer: Shu Ming shum...@linux.vnet.ibm.com Gerrit-Reviewer: Zhou Zheng Sheng zhshz...@linux.vnet.ibm.com Gerrit-Reviewer: oVirt Jenkins CI Server ___ vdsm-patches mailing list vdsm-patches@lists.fedorahosted.org https://lists.fedorahosted.org/mailman/listinfo/vdsm-patches
Change in vdsm[master]: remoteFileHandler: config logging in spawned process
Zhou Zheng Sheng has posted comments on this change. Change subject: remoteFileHandler: config logging in spawned process .. Patch Set 7: (1 inline comment) Thanks Allon, Dan. File vdsm/storage/remoteFileHandler.py Line 46: import misc Line 47: import fileUtils Line 48: import zombieReaper Line 49: from vdsm import constants Line 50: Question 1 Answer: I start a Python interpreter and import the misc, fileUtils, zombieReaper module as it does in remoteFileHandler. The output of ps aux is as follow. root 31367 0.3 0.1 220496 11668 pts/1S+ 10:46 0:00 python Then import the constants module and ps again, the VSZ and RSS do not increase, which means the constants module takes less than 1KB memory. Question 2 Answer: Yes, when multiple process append to the same log file, the messages will be mixed, Python only synchronize the log output within a process but not inter-process. We can have a thread listening on a port and deliver all the remoteFileHandler process logging messages to that port, then the listening thread write the messages to one log file. We can use logging.handler.DatagramHandler or SyslogHandler to do this. To use DatagramHandler, we have to create and manage the listening thread in VDSM service, to use SyslogHandler, we can configure it to deliver remoteFileHandler messages to /var/log/vdsm/remoteFileHandler.log . When debugging, I found the log messages from the remoteFileHandler process is very useful. I think the most simple solution is configure syslog to deliver the remoteFileHandler messages to a log file in /var/log/vdsm/remoteFileHandler.log. And in the main body of remoteFileHandler, do a basic configure to deliver the messages to syslog. If you think the proposal is helpful I can submit new patch set using syslog, or I will abandon this patch. Line 51: Line 52: # Crabs are known for their remote process calls Line 53: LENGTH_STRUCT_FMT = Q Line 54: LENGTH_STRUCT_LENGTH = calcsize(LENGTH_STRUCT_FMT) -- To view, visit http://gerrit.ovirt.org/9182 To unsubscribe, visit http://gerrit.ovirt.org/settings Gerrit-MessageType: comment Gerrit-Change-Id: I1b3b9e2837d632c7532fcd8f7306ed50b0865b5c Gerrit-PatchSet: 7 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Zhou Zheng Sheng zhshz...@linux.vnet.ibm.com Gerrit-Reviewer: Allon Mureinik amure...@redhat.com Gerrit-Reviewer: Dan Kenigsberg dan...@redhat.com Gerrit-Reviewer: Federico Simoncelli fsimo...@redhat.com Gerrit-Reviewer: Mark Wu wu...@linux.vnet.ibm.com Gerrit-Reviewer: Saggi Mizrahi smizr...@redhat.com Gerrit-Reviewer: ShaoHe Feng shao...@linux.vnet.ibm.com Gerrit-Reviewer: Shu Ming shum...@linux.vnet.ibm.com Gerrit-Reviewer: Zhou Zheng Sheng zhshz...@linux.vnet.ibm.com Gerrit-Reviewer: oVirt Jenkins CI Server ___ vdsm-patches mailing list vdsm-patches@lists.fedorahosted.org https://lists.fedorahosted.org/mailman/listinfo/vdsm-patches
