If you want to stay with index/data for the database, I think I might go back
and see if I can finish up a database.c replacement with MySql. What I
don't have is the statistics for a large email server (mine is very small).
I wrote a small C program that went through all the Berkley databases
>Just to chime in, the database.c code is very clean. The only problem
I
>have had with Berkeley is that if the code or system crashes, most of the
>time, the database is corrupted and recover does not help sometimes and
I
>have to revert to a backup. The code has become a lot more
I'm away on vacation right now and casually browsing the changes. I'll give
it a good thorough testing when I'm back at my desk after the weekend, but
so far I'm quite impressed. It looks like the changes really follow not just
the coding style, but the *design* style of the existing Citadel
Just to chime in, the database.c code is very clean. The only problem I
have had with Berkeley is that if the code or system crashes, most of the
time, the database is corrupted and recover does not help sometimes and I
have to revert to a backup. The code has become a lot more stable so not as
>Not sure if regression, but if you try to sent a message with a blank body,
>webcit coredumps. Just checked for empty message text and skipped trying
to
>output.
I wasn't able to reproduce the problem but the patch looks like it will add
some safety so I pulled it in anyway.
Right, and we do table-level locking in the application before we even attempt
to touch the database, so that has worked pretty well.
I did end up using an alternate --prefix just to keep the build a little
more solid, but that seems to just change the symbol names internally without
actually
> 2023-06-20 15:02 from IGnatius T Foobar
>Subject: Re: Long term plans for phase-out of Berkeley DB?
>
> Heh. Most of the database layer is still pretty much the same code
>you wrote 22 years ago. You did a great job on that.
As I recall, I interpreted their
Heh. Most of the database layer is still pretty much the same code you wrote
22 years ago. You did a great job on that.
I've thought about switching to a SQL model. It would confer a lot of
benefits.
It would also open up a huge effort in getting all of the code converted
over to
> 2023-06-15 13:32 from IGnatius T Foobar
>Subject: Long term plans for phase-out of Berkeley DB?
>
> A bit of thinking out loud with regard to long term plans for Citadel
>Server.
>
> Berkeley DB seems to have gained a bad reputation in the open source
>community.
I see the merge request and will check it out. Thanks!
Oh! Ok, that's why I didn't see a merge request. Cool, that's exactly how
it should be. :)
In other news, I finally figured out why the server kept crashing (on this
site). Someone put a match string in their inbox rules that doesn't compile
to a regular expression. regcomp() fails, and
That is not the branch that was merged. It is the new one to actually add
support for all the flags. I ran into some issues testing so I am working
on it locally. Should be updating the branch and submitting soon. Adding
the flags is a big change (hopefully for the better), but touches a lot
I also see that the `Support_IMAP_Flagged_Deleted_Draft_Recent_flags` branch
is still there. I thought it was going to get deleted in the merge. Mind
if I delete it?
I also have to make sure I'm not "squashing" multiple commits so they all
get their own commit log messages.
Thanks for
I am completely new to Git and commits, so I did not realize there was a
standard :-P
Thanks for the feedback and access. I'll now shut up and code :-)
I'm assuming you're familiar with the industry-wide standard for how to write
a commit message (summary line of 50 characters or less, then if you need
to type more, skip a line and then make the longer description lines of 72
characters or less).
If you're doing that in the commit message,
Ok. I have gotten my feet wet, so now I have a couple more questions. I
am trying not to be annoying, but I seem to be running into things that I am
not sure how to handle. Just let me know if I should shut up and code :-)
As you have seen, I have been creating rather long, detailed commit
>From my experience using other gitlab installations, yes you have to create
a branch and then commit that branch, and then you can create a merge request.
When the merge is approved (and I haven't set any approval rules so we might
have to experiment there) there is a "delete source branch"
Did not mean to attach the patch in the last message.
It looks like Merge requires you to create a branch to merge. Do you have
any naming conventions you would like to use for branch names or is there a
different way you want to submit code?
> Thu Jun 01 2023 09:32:30 PM EDT from
GitLab looks nice. I figured out how to clone and I created a patch. I
will go and try to use merge.
0001-IMAP_Flag_Clearing_and_User_Purge_Vist_Records.patch
Description: Binary data
Ok, I see you logged in to GitLab. Your account is registered and added to
the Citadel group as a developer. You can probably add your external email
address and make it primary if you want to at this point.
At this point you *should* be able to submit Merge Requests for patches.
This is
>fixed a couple of defects with the existing code that I am just going to
>include with the new flag support. Small but if you want me to submit
>separate, let me know.
I'd like it if you did that, please. If for no other reason, it would be
a great way for us both to get used to
I'm not surprised. That code has seen a lot of tweaking by a number of people
and it could use some attention.
The underlying data formats are at the front of my attention right now because
I'
m writing a utility to convert a 32-bit database to 64-bit, and it's uncovering
a number of
I am testing a complete replacement for the Visit handling. Looking good so
far, but since we don't have a set of tests to check for regression, I am
just running it and trying to hit as many code paths as I can. In order
to add all the IMAP flags, I had to pretty much rewrite the related code.
>problem. The server will operate fine for about an hour then it starts
>deleting users without me being in the interface. The stranger part is
that
This room isn't really for support issues, but you might want to check the
time and date on your system.
Also be sure that you're
>Noticed flags were getting set wrong in Outlook and in WebCit. I thought
I
>had introduced the bug(s). Finally realized that the bugs were in the
>release and had nothing to do with my code. Spent way too much time
>debugging my code :-)
I've seen it malfunction but never
>Now to the issue: How do you want to support the change to the structure
and
>falling back to older versions of the mail server. My implementation reads
First of all, I love your idea and we should definitely do it. If I'm reading
your description correctly, it could potentially mean
All right, I've applied those patches. I'm giving some consideration to
deploying a GitLab instance so we can have an automated testing and deployment
pipeline; if we do that then you can just have an account and generate merge
requests.
Easy Install isn't in Git at all; if you have
Phil Slack
> 2023-05-03 16:12 from HarlowSolutions
By what name do you want to be credited in the documentation?
(oh and thanks for the patch!)
libcitadel and webcit use the GNU Automake build system. That's why there's
a bootstrap program; it sets up the build environment on your build machine
so you can continue.
citadel (server), textclient, and webcit-ng use a much simpler build system,
one that doesn't rely on the GNU
The bootstrap was the part I was missing. Thanks.
For webcit, I am glad to hear that it is getting rewritten. At one point, I
tried to fix the calendar so it would work as an internet calendar in
Outlook, but after a bunch of work, it was not worth it. Got it talking to
Outlook, but then I
The patch looks good, and I'll go ahead and see if it applies cleanly. The
important thing is that it doesn't try to change too many things at the same
time, and you've documented what it is intended to do. That's really really
good.
By the way -- in case you haven't figured it
Submitting Code:
What you said makes very good sense. To try out submitting code, I cloned
down the citadel release, created a branch and made some updates to fix some
WebCit page issues and cleaned up a little formatting. Seemed like a simple
change to submit with low impact for a first try.
Ok, let me see if I can answer these questions one at a time.
git: at any given time, you'll want to be keeping up with the code in the
git repo. If you are developing against a released version of the code you
might have trouble integrating your changes.
That having been said, no
Thanks. Looking forward to it.
Great! I just want to answer quick and let you know I'll read through this
soon -- I'm super busy at work this week but I wanted to let you know these
questions *will* be answered. Give me a bit and I'll go through it. Thanks
for being interested in contributing!
>wtf, why do you want to break the universe?
Heh. Sometimes there are days when it feels like the universe is not worth
saving :)
wtf, why do you want to break the universe?
Mon Mar 13 2023 06:27:26 EDT from noreply Subject: Daily commit digest for Citadel
cff70a055 Art Cancro 2023-03-12 Implemented an algorithm that can correctly divide by zero.
(No bonus points for guessing correctly that these aren't actually significant
releases and I'm really just testing a CI/CD pipeline.)
Sometimes a bit of code just "feels good" and you know you got it right.
The "Delete" button in the forum view, for example, which just got committed.
We only display the button if we know we have permission to delete messages
in the current room, which is a permission indicated when we fetch
Those who have followed the development of the server for many years will notice an intent here. Look at how little happens between receiving a signal and exiting the server process. In the past, we made an effort to clean everything up. We cleanly logged out every session, each module
Sat Jun 11 2022 13:12:07 EDT from IGnatius T Foobar Subject: Re: Notes on new build system
I'm going to need some help with QA on FreeBSD and I'm hoping that LadySerenaKitty is willing to continue offering a bit of time in that area.
The running theory is that I am going to continue
I do want to test on ARM hardware other than a Pi, but that will have to come
later. I keep forgetting that Citadel is, for some reason, staggeringly popular
on Raspberry Pi. I don't know why; all I know is that when I break the build,
I hear about it. And I've finally nailed down that
Oh boy.
That was a really long and painful day of troubleshooting. After updating
all of my SSL server code to the latest practices, I was no longer able to
establish SSL connections to Uncensored EXCEPT on HTTPS. So no encrypted
IMAP, no encrypted XMPP, everything just threw a cipher
After a lot of effort I am still unable to reproduce the problem you are
describing. Are you able to watch the citserver log while you change
an account's email aliases? I would like to see what it "thinks" it's
doing.
On 1/5/22 14:40, Art Cancro wrote:
Is this the correct summary of
Is this the correct summary of the problem?
On 12/31/21 14:36, s3cr3to wrote:
But sadly, the aliases are not preserved neither from the text client,
and even less from webcit; every time I edit a new alias; it
eliminates the previous ones: It go through the first alias and places
it as
The sendcommand binary should be in /usr/local/citadel in the container's
filesystem.
But if you *need* sendcommand for something, I should probably know what
it is, so we can provide a better way to do it.
For those of you following along ... the reason I am enduring all of this
extreme frustration of working with the WebCit Classic code base -- which
has been rendered unmaintainable by bloody acres of hash tables and callback
functions -- is because I want to make it work with the ACME HTTP-01
What a fun evening of hacking!
Every time I discover something like this -- in this case, scrollIntoView()
-- it affirms that I made the right choice building a new WebCit from scratch.
Web development is such a different beast than it was when I started WebCit
25 years ago. The browser has so many things built-in that
Is everyone enjoying the commit log? Hehe
WebCit-NG is front and center again :)
Ooooh, yeah that's right; mail to rooms with spaces in the name, working
again now :)
Test sept-19, 2:07PM... using Firefox.
> Fri Sep 17 2021 08:46:41 PM EDT from s3cr3to Subject: Re: Citadel 938
>
>
>
>(I don't see the Test #1 arrive yet)
>
>Test #2, But now I see the message below displayed correctly; I wonder why
>was the cause (I delete firefox Cache way before
(I don't see the Test #1 arrive yet)
Test #2, But now I see the message below displayed correctly; I wonder why
was the cause (I delete firefox Cache way before my test #1 today)
> Sat Sep 11 2021 04:34:10 PM EDT from IGnatius T Foobar Subject: Citadel
>938
>
>
>All righty
Citadel 935 is now available via Easy Install, and by tomorrow it will be
available on Docker Hub.
Ok, it's working well now, just needs a few cosmetic cleanups and we'll be
ready to roll.
>Or a new feature? (let's hope not)
I certainly hope not! :)
It's the code in the server that implements expansion of addresses in the
global alias list. When an address in the alias expansion is larger than
a certain number of characters (feels like around 30 or so) it clobbers
Or a new feature? (let's hope not)
On 9/8/21 9:21 AM, noreply wrote:
This is an automated email from the git hooks/post-receive script. It was
generated because a ref change was pushed to the repository containing
the project "Citadel Server, WebCit, utilities".
The branch, master has been
>Yippee-Ki-Yay My Friend!
There's one more bug to squash, a segfault that appears on certain address
combinations. After that's fixed we will go straight to release.
Yippee-Ki-Yay My Friend!
https://www.youtube-nocookie.com/embed/fxBOlLFz6NU
On 9/6/21 4:21 PM, noreply wrote:
This is an automated email from the git hooks/post-receive script. It was
generated because a ref change was pushed to the repository containing
the project "Citadel Server, WebCit,
For me, you just saying that plans had not changed was assurance enough :)
The above is kind of a big deal :)
This means that the code available in the Docker container is going to be
exactly the same as the code available through Easy Install. You wanted
assurance
that Easy Install isn't going away -- here it is. It's all part of the same
build chain now:
The latest build has AMD64/i386/ARMv7 thanks to "buildx" which does all of
the cross-compiling for you. This is very cool, and now I can use my Pi for
testing instead of keeping it tied up in the build pipeline. buildx *does*
support 64-bit ARM, and originally I included it in the build.
Well the multi arch pull fails for 64bit as it cant find it in the manifest. Sort of what i expected to be honest. Unlike forcing a specific architecture during the pull where it just complains ( but runs ) here it says "screw you its not there" and hard stop. ( in my case its
I heard of people trying to run docker on lxc. Didn't work out too well for them. :)
Tue Aug 24 2021 05:50:31 PM EDT from IGnatius T Foobar Subject: Re: (running-docker-on-proxmox)
Yes, and it appears that the easiest way to do that is just to run an ordinary Ubuntu VM on your ProxMox
I didn't pay a lot of attention, but doesn't Proxmox support Docker containers? Or is it just Kurbernettes - or whatever...
Proxmox supports containers using LXC, not Docker. It's still containers, but LXC containers are really more intended to be lightweight virtual machines that share a
Glad to read the blog and be able to learn a lot.
On 8/12/21 8:06 PM, IGnatius T Foobar wrote:
Do you guys find it helpful and/or interesting that I sort of blog the details
of my development effort here? I mostly do it to organize my thoughts but
it would be nice to know that someone
I go for run Docker in a VM, from:
https://danthesalmon.com/running-docker-on-proxmox/
"/*Run Docker in an LXC container*//
//Security warning: This configuration offers very little, if any
security to segment the contents of the container from the Proxmox
host. This method should
I read the posts. So, yes.
Thu Aug 12 2021 22:06:57 EDT from IGnatius T Foobar Subject: New design pattern in WebCit
Do you guys find it helpful and/or interesting that I sort of blog the details of my development effort here?
It's working exceptionally well, and I am seriously considering switching
from virtual machines to containers for all of my workloads. They all run
on Linux anyway.
Cool. Makes it easier for our side of the fence. Ill try one of those pulls tomorrow.
Mon Aug 23 2021 01:27:06 PM EDT from IGnatius T Foobar Subject: Re: New Docker Images
First things first: I figured out how to do a "multiarch" build, so there will no longer be different tags for
Might as well have this conversation here.
First things first: I figured out how to do a "multiarch" build, so there
will no longer be different tags for different architectures. "latest" now
supports AMD64, i386, and ARMv7. I built a script to push nightly builds,
so hopefully we've got
The new location is:git://git.citadel.org/citadel
And of course we still have the gitweb browser at: https://code.citadel.org
I have emailed the new SSH location to those who have write access.
On 8/12/21 12:18 PM, IGnatius T Foobar wrote:
In case anyone is tracking the Citadel git repository -- be advised that
I am moving it. I am tired of maintaining an "/appl" hierarchy on my server;
this was done in the distant past before /opt and /usr/local came into
widespread
standard
>Thank you very much for your work; at the moment I am out of the office,
>I think for this week. (I need to finish a project by mid-September, I
>will try to escape my earring at night)
>
>As soon as I get it, I restore/cleanup the VM/Debian to test it.
Thanks. So far, it is
Thank you very much for your work; at the moment I am out of the office,
I think for this week. (I need to finish a project by mid-September, I
will try to escape my earring at night)
As soon as I get it, I restore/cleanup the VM/Debian to test it.
On 8/9/21 10:46 PM, IGnatius T Foobar wrote:
Thanks for the information.
On 8/5/21 6:27 PM, IGnatius T Foobar wrote:
>* VM = OS + Docker + Citadel
Right. It should be ridiculously easy. I'm discovering a lot of interesting
things about Docker, which are helping to make things nice and slick:
--> I discovered the
* VM = OS + Docker + Citadel
On 8/4/21 11:47 PM, s3cr3to wrote:
Year before I tried a little with Ubuntu; and a little using Synology
which allows using Docker. But I'm more interested in using a
dedicated VM* for Citadel.
So, I seem to remember that you use Ubuntu to develop Citadel;
Then... Should I use a VM with Ubuntu server to use Docker + Citadel?
I don't really know which is the right distribution for Docker.
2021 https://kuberty.io/blog/best-os-for-docker/
2021
Lots of great progress tonight on the Docker container. As expected, much
of the work that went into the AppImage was reusable in Docker, and the Docker
version is running quite well already. It doesn't feel as "fragile" as the
AppImage did, and I think it's going to serve us better in the
>So, do I stop my tests with the AppImage and wait for the thingy* in Docker?
Let's put it on hold for now. The goal is the same, but the package format
is different.
We've learned a lot over the last few months, and in developing the AppImage
we solved a LOT of problems that were
So, do I stop my tests with the AppImage and wait for the thingy* in Docker?
*No disrespect intended.
On 8/3/21 4:29 PM, noreply wrote:
This is an automated email from the git hooks/post-receive script. It was
generated because a ref change was pushed to the repository containing
the project
The "series of messages" leads to a sad unraveling.
Well, to a more complex one, I had stopped studying Docker as soon as I
saw the AppImage, I will have to resume its study and use it.
To date, I had barely played with Docker; and like a good Alzheimer's
trainee, I forgot about it in a
The only good thing about not seeing your emails for 3 weeks is that I
am binge reading them.
WOW!
On 7/28/21 3:56 PM, IGnatius T Foobar wrote:
s3cr3to: this is all towards your use case, so hold tight, it'll be ready
soon!
s3cr3to: this is all towards your use case, so hold tight, it'll be ready
soon!
Ok, with the AppImage release now completed, my next task will be to add
the global email alias table. It won't be pretty, but it'll be accessible
for those who need it.
Sorry I was not very clear.
Currently on the server in production (deb install), I have several
accounts with many aliases.
So, if I would really like to have the version that preserves all these
aliases (from the old style) as in the V-card.
But, if they are going to be lost; then I will
>I assume that restoring the backup will preserve the ones that are
>currently there, am I correct?
What do you mean by "currently there"?
If you're talking about aliases that were in the user account already, with
the limited amount of space, then yes, those will be preserved
Good day Art
I was out of the office last week. I could not see the mails in
"Development", something happens with my Thunderbird at home.
Yes the global alias table works for me if with that I manage to place
all the aliases we handle.
I assume that restoring the backup will preserve the
The absence of the text client is definitely still something that is missing from the appimage. I'm trying to figure out what happens when the same AppImage is called multiple times on the same host. Does it consume more memory or does it share? If it shares, I could add a calling mode that
Greetings Art, hope you are well.
Although I have not been able to do much testing, I have a couple of
questions:
1. Do you plan to increase the amount of aliases to support per mail
account? This is so far the only detail that stops me from migrating my
server to the current one.
2. How
Tell you what ... since your bug reports tend to be good quality and well
triaged, I'm going to just give you access to the "Citadel Issue Tracker"
wiki room. In the distant past we had general bug reporting, but it quickly
filled up with poorly researched/reported bugs, and with feature
Congratulations!
So, now I can test it and report bugs or missing features in Citadel?
(and what may be wrong with the App).
I'm too chatty when reporting bugs, do you have a special bug report
form to use to make it easier for you to follow up?
On 5/21/21 12:31 PM, IGnatius T Foobar
Good day,
Just a small observation with the year 2020.
May 21 10:18:55 em2 webcit[27474]: *WebCit 931*
May 21 10:18:55 em2 *webcit*[27474]: Copyright (C) 1996-_*2020*_ by
the citadel.org team
...
May 21 10:18:55 em2 citserver: *** Citadel server engine ***
May 21 10:18:55 em2
>Is there any way to list the size of the mailboxes of each account and
>maybe including the size of their folders?
Not inside of Citadel. You could probably find an IMAP tool that does that.
Greetings Art.
Is there any way to list the size of the mailboxes of each account and
maybe including the size of their folders?
I can't figure out where there is 71GB of space, of course it is
possible that there are huge junk files. And in most of the mailboxes I
set a time in months to
Excellent. It's too bad that Berkeley DB needs that kind of tuning. Perhaps
someday when 32-bit systems are a distant memory we will move to LMDB.
It works!
Compacted from 91G to 71G
From 4PM to 6:26pm
May 19 15:00:08 em2 citserver[1694]: db: compacting database 8
...
May 19 18:26:13 em2 citserver[1694]: housekeeping: WARNING:
housekeeping loop has not run for 206 minutes. Is something stuck?
May 19 18:26:14 em2
OK, I'll try with this values.
The *first 3* are from my server in production.
And I will try with the *double* in "set_cachesize" of what is suggested
in the document.
# cat /usr/local/citadel/data/DB_CONFIG
*set_lk_max_locks 4000**
**set_lk_max_lockers 4000**
>*May 17 14:21:06 em2 citserver[25444]: db: BDB3017 unable to
>allocate space from the buffer cache*
>*May 17 14:21:06 em2 citserver[25444]: db: compact: Cannot allocate
>memory*
You didn't recover any disk space because it crashed. This means we have
to put some
101 - 200 of 1530 matches
Mail list logo
