Re: [systemd-devel] Udevd and dev file creation
While reading the code I came across :- struct Manager struct Worker Can you please provide an overview on these, I did not get a clear cut explanation on these structs, the code only has its member variables. Also what is the variable 'arg_children_max' Why is it specifically set to 8 + no_of_cpus*2 ? Regards Nishant Nayan On Tue, 1 Feb 2022, 14:13 Greg KH, wrote: > On Tue, Feb 01, 2022 at 10:08:04AM +0530, Nishant Nayan wrote: > > Thanks, I will check that out. > > Is there any site where I can gain in depth code level knowledge of > > systemd-udevd? > > The code is all there for you to read and understand directly. What > specific questions did you have that you did not understand after > reading it? > > thanks, > > greg k-h >
Re: [systemd-devel] Udevd and dev file creation
Just wanted to know what will happen. Nishant On Tue, 1 Feb 2022, 16:14 Greg KH, wrote: > On Tue, Feb 01, 2022 at 04:04:01PM +0530, Nishant Nayan wrote: > > One thought > > Is it advisable to turn off systemd-udevd if I am sure that I won't be > > adding /removing any devices to my server. > > Why would you want to do that? > > > Or udev also does some work while rebooting? > > When booting, yes. >
Re: [systemd-devel] Udevd and dev file creation
One thought Is it advisable to turn off systemd-udevd if I am sure that I won't be adding /removing any devices to my server. Or udev also does some work while rebooting? Regards Nishant Nayan On Tue, 1 Feb 2022, 14:13 Greg KH, wrote: > On Tue, Feb 01, 2022 at 10:08:04AM +0530, Nishant Nayan wrote: > > Thanks, I will check that out. > > Is there any site where I can gain in depth code level knowledge of > > systemd-udevd? > > The code is all there for you to read and understand directly. What > specific questions did you have that you did not understand after > reading it? > > thanks, > > greg k-h >
Re: [systemd-devel] Udevd and dev file creation
Thanks, I will check that out. Is there any site where I can gain in depth code level knowledge of systemd-udevd? Regards Nishant Nayan On Mon, 31 Jan 2022, 17:13 Lennart Poettering, wrote: > On So, 30.01.22 17:14, Nishant Nayan (nayan.nishant2...@gmail.com) wrote: > > > I have started reading about udevd. > > I was trying to find out if there is a way to play with udev without > > plugging in/out any devices. > > Is there a way to trigger a uevent without plugging in devices? > > use "udevadm trigger" to fire uevents for existing devices. > > Or create new, synthetic virtual devices during runtime, for example > via "losetup". > > Lennart > > -- > Lennart Poettering, Berlin >
Re: [systemd-devel] Udevd and dev file creation
I have started reading about udevd. I was trying to find out if there is a way to play with udev without plugging in/out any devices. Is there a way to trigger a uevent without plugging in devices? Regards Nishant Nayan On Sun, 30 Jan 2022, 17:13 Nishant Nayan, wrote: > Oops. > Thanks for pointing that out, I just pressed the reply button, didn't > notice it. > Resending now. > > > Regards > Nishant Nayan > > On Sun, 30 Jan 2022, 17:12 Greg KH, wrote: > >> On Sun, Jan 30, 2022 at 05:07:52PM +0530, Nishant Nayan wrote: >> >> >> >> For some reason you sent this only to me, which is a bit rude to >> everyone else on the mailing list. I'll be glad to respond if you >> resend it to everyone. >> >> thanks, >> >> greg k-h >> >
[systemd-devel] Udevd and dev file creation
Hi, Does creating a dev file in /dev creates a uevent? Does a dev file creation considered as a "device attached" to the machine? Regards Nishant Nayan
[systemd-devel] which processes writes to journald ?
Does journald logs only for systemd services or for any process? I am kinda confused about difference between systemd units and userspace processes, are both same ? Do we have any classification so as to which process will send logs into which socket? For example :- Which processes sends logs into 1) /dev/log 2) /run/systemd/journal/stdout 3) /run/systemd/journal/socket ? Nishant
[systemd-devel] Why journal logs are stored in binary?
I searched as to why journal logs are in binary format unlike plaintext syslog, one answer was 'for faster log scanning', how does binary format helps in faster scanning and what are the advantages of it? Nishant
[systemd-devel] Use of systemd-cat
I read systemd-cat manpage and its functiinality, how exactly can it be useful for logging and debugging, any example would be helpful. Nishant
Re: [systemd-devel] Why systemd-journald is creating '/var/log/journal/machine_id" directory when Storage=auto
Awesome, thanks! Also, where can I find the code section where services And kernel logs to journald? After tracing from 'main' at journald.c I came across the part where various sockets are opened and validated (/run/systemd/journal/stdout, /dev/kmsg, /dev/log ) for journald to listen to logs of systemd services and kernel. That is the server side part. Where can I find the client side journald code where services and kernel sends their logs to journal. Nishant On Sun, 29 Aug 2021, 18:39 Michael Chapman, wrote: > On Sun, 29 Aug 2021, Nishant Nayan wrote: > > Also I was wondering where in the code does journald.config file changes > > are parsed? > > For example in the above code , the line :- > > if (s->storage == STORAGE_PERSISTENT) > > Here, s->storage corresponds to 'Storage' option of conf file right? > > How is it getting set when we edit the conf file? > > The configuration files are loaded in server_parse_config_file(). The > actual code that maps keys in the files to variables in the program is > generated by gperf [1] from the journald-gperf.gperf source file. > > [1] https://www.gnu.org/software/gperf/manual/html_node/index.html > > > Also, on doing "ls /run/log/journal/machine_id/" > > I can see output as following > > .journal > > .journal > > . > > . > > . > > system.journal > > > > Is 'system.journal' is the currently active journal and rest are archived > > journals? > > That's correct. >
Re: [systemd-devel] Why systemd-journald is creating '/var/log/journal/machine_id" directory when Storage=auto
Also I was wondering where in the code does journald.config file changes
are parsed?
For example in the above code , the line :-
if (s->storage == STORAGE_PERSISTENT)
Here, s->storage corresponds to 'Storage' option of conf file right?
How is it getting set when we edit the conf file?
Also, on doing "ls /run/log/journal/machine_id/"
I can see output as following
.journal
.journal
.
.
.
system.journal
Is 'system.journal' is the currently active journal and rest are archived
journals?
Nishant
On Sun, 29 Aug 2021 at 17:56, Michael Chapman
wrote:
> On Sun, 29 Aug 2021, Nishant Nayan wrote:
> > I was looking into the code of systemd-journald and found this (in
> > system_journal_open() ) :-
> >
> > if (!s->system_journal && IN_SET(s->storage, STORAGE_PERSISTENT,
> > STORAGE_AUTO) && (flush_requested || flushed_flag_is_set())) {
> >
> > /* If in auto mode: first try to create the machine
> > * path, but not the prefix.
> > *
> > * If in persistent mode: create /var/log/journal and
> > * the machine path */
> >
> > if (s->storage == STORAGE_PERSISTENT)
> > (void) mkdir_p("/var/log/journal/", 0755);
> >
> > (void) mkdir(s->system_storage.path, 0755);
> >
> > fn = strjoina(s->system_storage.path, "/system.journal");
> [...]
> >
> > Also after reading the comment, how is it possible to create
> > '/var/log/journal/machine_id' without creating the prefix? I am assuming
> > '/var/log/journal' is the prefix .
>
> Sorry, I actually misread your question here.
>
> In Storage=auto mode, no attempt to create the /var/log/journal directory
> is made. If it doesn't exist, then the persistent journal simply fails to
> be opened.
>
[systemd-devel] Why systemd-journald is creating '/var/log/journal/machine_id" directory when Storage=auto
I was looking into the code of systemd-journald and found this (in
system_journal_open() ) :-
if (!s->system_journal && IN_SET(s->storage, STORAGE_PERSISTENT,
STORAGE_AUTO) && (flush_requested || flushed_flag_is_set())) {
/* If in auto mode: first try to create the machine
* path, but not the prefix.
*
* If in persistent mode: create /var/log/journal and
* the machine path */
if (s->storage == STORAGE_PERSISTENT)
(void) mkdir_p("/var/log/journal/", 0755);
(void) mkdir(s->system_storage.path, 0755);
fn = strjoina(s->system_storage.path, "/system.journal");
Here, system_storage.path is set to "strjoin("/var/log/journal/",
SERVER_MACHINE_ID(s));" in previous function call.
As far as I understood its saying to create '/var/log/journal' directory
when storage is set to 'persistent'.
But in either of the cases (persistent or auto) why is it creating
'/var/log/journal/machine_id' directory ( (void)
mkdir(s->system_storage.path, 0755); ) ??
'auto' will store logs persistently if '/var/log/journal' is created
beforehand or else logs will be written in '/run/log/journal' .
For 'auto' it should not create '/var/log/journal/machine_id' directory
right?
Also after reading the comment, how is it possible to create
'/var/log/journal/machine_id' without creating the prefix? I am assuming
'/var/log/journal' is the prefix .
Re: [systemd-devel] why log_set_prohibit_ipc() is set in journald
So then where does journald logs its own messages if he wants to? On Fri, 27 Aug 2021, 13:37 Lennart Poettering, wrote: > On Fr, 27.08.21 10:01, Mantas Mikulėnas (graw...@gmail.com) wrote: > > > On Fri, Aug 27, 2021, 08:52 Nishant Nayan > > wrote: > > > > > I have just started to learn journald and in its main function (in > > > journald.c) I encountered a function call "log_set_prohibit_ipc(true);" > > > In systemd source, I can see the declaration in src/basic/log.h:/* > > > > > > If turned on, then we'll never use IPC-based logging, * i.e. never log > to > > > syslog or the journal. We'll only * log to stderr, the console or kmsg > > > */void log_set_prohibit_ipc(bool b); > > > > > > I did not get this because Journald not writing to journal itself by > > > default is strange, isn't it? > > > What is the reason behind it? > > > > > > > My understanding is that the point isn't to prevent logging to journal, > but > > to prevent logging *through IPC* specifically, i.e. make sure journald > > doesn't try to create loopback connections to its own sockets. The > journald > > daemon is single-threaded, so if it tries to connect to itself, it'll > > deadlock. > > > > But also if journald wants to log a critical error (e.g. running out of > > space or something like that), then it can't really *rely* on journal > still > > working... > > > > Afaik, messages written to kmsg will be imported back into the journal > > anyway, but that happens asynchronously so it's fine. > > The above describes exactly how it is, and why journald turns of > logging via IPC. journald should not be a client to itself. > > Lennart > > -- > Lennart Poettering, Berlin >
[systemd-devel] why log_set_prohibit_ipc() is set in journald
I have just started to learn journald and in its main function (in journald.c) I encountered a function call "log_set_prohibit_ipc(true);" In systemd source, I can see the declaration in src/basic/log.h:/* If turned on, then we'll never use IPC-based logging, * i.e. never log to syslog or the journal. We'll only * log to stderr, the console or kmsg */void log_set_prohibit_ipc(bool b); I did not get this because Journald not writing to journal itself by default is strange, isn't it? What is the reason behind it? Nishant
Re: [systemd-devel] How does journald talks to other services?
Got it. What do we mean by "indexing" of logs for faster lookup? , Journlad does this I have seen this word in docs but no clear explanation, is it related to addition of timestamp, process_name, log_priority (info,debug...) to the log messages ? On Tue, 24 Aug 2021 at 11:54, Lennart Poettering wrote: > On Di, 24.08.21 09:11, Nishant Nayan (nayan.nishant2...@gmail.com) wrote: > > > So what are the cases where syslog forwards logs to journal? > > Is there a case where both journal and syslog end up sending same logs to > > each other ( like a cycle ) resulting in duplicate logs? > > systemd does not pick up messages from another syslog service, only from > syslog clients. Thus, there is no loop. > > Lennart > > -- > Lennart Poettering, Berlin >
Re: [systemd-devel] How does journald talks to other services?
So what are the cases where syslog forwards logs to journal? Is there a case where both journal and syslog end up sending same logs to each other ( like a cycle ) resulting in duplicate logs? Nishant On Mon, 23 Aug 2021, 14:02 Mantas Mikulėnas, wrote: > On Mon, Aug 23, 2021, 11:19 Nishant Nayan > wrote: > >> I was using logger command to see if the logs goes to journal, and it >> does, it goes both in /var/log/messages (owned by syslog) and journal, how >> is it happening? Is it because journal listens to /dev/log ? >> > > Journald listens to /dev/log and writes messages to its .journal files. > Then a syslog daemon (rsyslogd or syslog-ng) receives the same messages > *from* journald, in one of two ways, and writes them to /var/log/messages: > > a) The syslog daemon directly reads messages with full metadata from > .journal files (e.g. in rsyslogd this is the imjournal module); > > or b) The syslog daemon listens on a completely separate socket in /run, > and journald forwards all messages to that socket (without metadata) using > the traditional syslog protocol. > > The following is from systemd-journald.socket >> [Socket] >> ListenStream=/run/systemd/journal/stdout >> ListenDatagram=/run/systemd/journal/socket >> ListenDatagram=/dev/log >> >> Also can we edit 'systemd-journald.socket ' so as to not listen to >> /dev/log ? Just for seeing its behaviour. >> I tried by commenting out and removing 'ListenDatagram=/dev/log' and >> restarted the socket and journal service, but the logger log is still >> displayed in journal >> > > Technically that should work? But don't use it for other reasons except > testing, I'd say... > > Did you systemctl daemon-reload? > > Is /dev/log a real socket or a symlink? (In later systemd versions it's a > symlink and the real socket is in /run.) > > If it's a real socket, does it get re-created after 'rm'? > > >> >> >> Nishant >> >> On Fri, 20 Aug 2021 at 16:43, Mantas Mikulėnas wrote: >> >>> On Fri, Aug 20, 2021 at 2:11 PM Mantas Mikulėnas >>> wrote: >>> >>>> On Fri, Aug 20, 2021 at 2:10 PM Nishant Nayan < >>>> nayan.nishant2...@gmail.com> wrote: >>>> >>>>> Regarding the below point : >>>>> c) The service prints to stdout/stderr, but systemd attaches the >>>>> service's stdout/stderr to a pipe which is read by journald (using >>>>> sd_journal_stream_fd(3) from libsystemd). See [Service] StandardOutput= in >>>>> systemd.service(5). >>>>> >>>>> I did not see StandardOutput field in [Service] sections of a service >>>>> file, for example sshd.service, but its logs are visible in journalctl. >>>>> Is it by default piped to journal and we need to explicitly mention it >>>>> (StandardOutput=) only when we want to redirect it somewhere else? >>>>> >>>> >>>> StandardOutput=journal is the default setting. >>>> >>> >>> And, actually, sshd doesn't write its messages to stdout anyway – it >>> uses syslog() via /dev/log; most daemons do. >>> >>> -- >>> Mantas Mikulėnas >>> >>
Re: [systemd-devel] How does journald talks to other services?
yes, I did daemon-reload and restarted the service and socket /dev/log is a real socket , it gets re-created on rm. Although, i have another system where /dev/log is a symlink On Mon, 23 Aug 2021 at 14:02, Mantas Mikulėnas wrote: > On Mon, Aug 23, 2021, 11:19 Nishant Nayan > wrote: > >> I was using logger command to see if the logs goes to journal, and it >> does, it goes both in /var/log/messages (owned by syslog) and journal, how >> is it happening? Is it because journal listens to /dev/log ? >> > > Journald listens to /dev/log and writes messages to its .journal files. > Then a syslog daemon (rsyslogd or syslog-ng) receives the same messages > *from* journald, in one of two ways, and writes them to /var/log/messages: > > a) The syslog daemon directly reads messages with full metadata from > .journal files (e.g. in rsyslogd this is the imjournal module); > > or b) The syslog daemon listens on a completely separate socket in /run, > and journald forwards all messages to that socket (without metadata) using > the traditional syslog protocol. > > The following is from systemd-journald.socket >> [Socket] >> ListenStream=/run/systemd/journal/stdout >> ListenDatagram=/run/systemd/journal/socket >> ListenDatagram=/dev/log >> >> Also can we edit 'systemd-journald.socket ' so as to not listen to >> /dev/log ? Just for seeing its behaviour. >> I tried by commenting out and removing 'ListenDatagram=/dev/log' and >> restarted the socket and journal service, but the logger log is still >> displayed in journal >> > > Technically that should work? But don't use it for other reasons except > testing, I'd say... > > Did you systemctl daemon-reload? > > Is /dev/log a real socket or a symlink? (In later systemd versions it's a > symlink and the real socket is in /run.) > > If it's a real socket, does it get re-created after 'rm'? > > >> >> >> Nishant >> >> On Fri, 20 Aug 2021 at 16:43, Mantas Mikulėnas wrote: >> >>> On Fri, Aug 20, 2021 at 2:11 PM Mantas Mikulėnas >>> wrote: >>> >>>> On Fri, Aug 20, 2021 at 2:10 PM Nishant Nayan < >>>> nayan.nishant2...@gmail.com> wrote: >>>> >>>>> Regarding the below point : >>>>> c) The service prints to stdout/stderr, but systemd attaches the >>>>> service's stdout/stderr to a pipe which is read by journald (using >>>>> sd_journal_stream_fd(3) from libsystemd). See [Service] StandardOutput= in >>>>> systemd.service(5). >>>>> >>>>> I did not see StandardOutput field in [Service] sections of a service >>>>> file, for example sshd.service, but its logs are visible in journalctl. >>>>> Is it by default piped to journal and we need to explicitly mention it >>>>> (StandardOutput=) only when we want to redirect it somewhere else? >>>>> >>>> >>>> StandardOutput=journal is the default setting. >>>> >>> >>> And, actually, sshd doesn't write its messages to stdout anyway – it >>> uses syslog() via /dev/log; most daemons do. >>> >>> -- >>> Mantas Mikulėnas >>> >>
Re: [systemd-devel] How does journald talks to other services?
I was using logger command to see if the logs goes to journal, and it does, it goes both in /var/log/messages (owned by syslog) and journal, how is it happening? Is it because journal listens to /dev/log ? The following is from systemd-journald.socket [Socket] ListenStream=/run/systemd/journal/stdout ListenDatagram=/run/systemd/journal/socket ListenDatagram=/dev/log Also can we edit 'systemd-journald.socket ' so as to not listen to /dev/log ? Just for seeing its behaviour. I tried by commenting out and removing 'ListenDatagram=/dev/log' and restarted the socket and journal service, but the logger log is still displayed in journal Nishant On Fri, 20 Aug 2021 at 16:43, Mantas Mikulėnas wrote: > On Fri, Aug 20, 2021 at 2:11 PM Mantas Mikulėnas > wrote: > >> On Fri, Aug 20, 2021 at 2:10 PM Nishant Nayan < >> nayan.nishant2...@gmail.com> wrote: >> >>> Regarding the below point : >>> c) The service prints to stdout/stderr, but systemd attaches the >>> service's stdout/stderr to a pipe which is read by journald (using >>> sd_journal_stream_fd(3) from libsystemd). See [Service] StandardOutput= in >>> systemd.service(5). >>> >>> I did not see StandardOutput field in [Service] sections of a service >>> file, for example sshd.service, but its logs are visible in journalctl. >>> Is it by default piped to journal and we need to explicitly mention it >>> (StandardOutput=) only when we want to redirect it somewhere else? >>> >> >> StandardOutput=journal is the default setting. >> > > And, actually, sshd doesn't write its messages to stdout anyway – it uses > syslog() via /dev/log; most daemons do. > > -- > Mantas Mikulėnas >
Re: [systemd-devel] How does journald talks to other services?
Regarding the below point : c) The service prints to stdout/stderr, but systemd attaches the service's stdout/stderr to a pipe which is read by journald (using sd_journal_stream_fd(3) from libsystemd). See [Service] StandardOutput= in systemd.service(5). I did not see StandardOutput field in [Service] sections of a service file, for example sshd.service, but its logs are visible in journalctl. Is it by default piped to journal and we need to explicitly mention it (StandardOutput=) only when we want to redirect it somewhere else? Even in my self made service I did not mention StandardOutput=, but I can see the logs in journalctl. Is there a place where I can see .service file defaults ? Nishant On Fri, 20 Aug 2021 at 15:20, Mantas Mikulėnas wrote: > On Fri, Aug 20, 2021 at 12:31 PM Nishant Nayan < > nayan.nishant2...@gmail.com> wrote: > >> Hi, >> My query is how does systemd-journald talk to other services so that >> it stores their logs/output in journal files, which could be displayed >> using journalctl utlity. >> > > Journald doesn't talk to services, services talk to journald: > > a) The service uses the standard syslog(3) call to send basic messages > through the /dev/log socket, where journald (or a traditional syslogd) is > listening. > > b) The service uses sd_journal_print(3) from libsystemd to send structured > messages through /run/systemd/journal/socket (systemd-journald.socket). > Some frameworks, such as GLib, have their own implementations of this > protocol without needing libsystemd. > > c) The service prints to stdout/stderr, but systemd attaches the service's > stdout/stderr to a pipe which is read by journald (using > sd_journal_stream_fd(3) from libsystemd). See [Service] StandardOutput= in > systemd.service(5). > > d) Journald also reads kernel messages (dmesg) from the /dev/kmsg device. > Programs can actually write to /dev/kmsg to generate dmesg messages and > journald will capture them. > > -- > Mantas Mikulėnas >
Re: [systemd-devel] How does journald talks to other services?
Thanks, I will check them out Nishant On Fri, 20 Aug 2021 at 15:22, Lennart Poettering wrote: > On Fr, 20.08.21 15:01, Nishant Nayan (nayan.nishant2...@gmail.com) wrote: > > > Hi, > > My query is how does systemd-journald talk to other services so that > it > > stores their logs/output in journal files, which could be displayed using > > journalctl utlity. > > I am currently looking into systemd journal code to find this out but > so > > far no luck. > > Any suggestions would be appreciated. > > See docs: > > https://www.freedesktop.org/software/systemd/man/systemd-journald.service > > There's a list at the very beginning of the description there that > lists the 5 different ways how log messages are delivered to journald. > > Lennart > > -- > Lennart Poettering, Berlin >
[systemd-devel] How does journald talks to other services?
Hi, My query is how does systemd-journald talk to other services so that it stores their logs/output in journal files, which could be displayed using journalctl utlity. I am currently looking into systemd journal code to find this out but so far no luck. Any suggestions would be appreciated. Regards Nishant
