@s3cr3to
We're making progress. I was unable to get the debugger working from within the AppImage, but I was able to run some other tests.
For example, even on a fresh database with no email sent, I found that if I set up an RSS feed, it would also crash when it goes to pull that feed. This mea
Oh ... I should have thought of that. Instead of sending the VM, you can
just let me log in to it remotely. Why didn't I think of that?
That will be fine, and I will make an effort to give it a try this weekend.
I will probably do something like this: mount the AppImage by hand, jump
in
Dammit. It's showing the backtrace of an idle thread, not the thread that
crashed.
I don't suppose you have enough bandwidth to simply upload the entire VM?
Ok, good. Now look in /tmp and see if there are any files in /tmp that begin
with "citserver-backtrace". Please post their contents or upload them
somewhere.
Check https://www.citadel.org/appimage.html for new AppImage builds.
This one doesn't *fix* any server crashes, but when the server does crash
it will save a backtrace in /tmp that you can upload here, so we can figure
out what's going wrong.
Hopefully. :)
Just a heads up to mention that crash reporting (particularly in AppImage
builds) is still the #1 priority. The previous bunch of commits were just
some brainless sweeping of the floor to keep me occupied while multitasking.
All right, exit code 139 means the server exited on signal 11, which means
that the AppImage supervisor did the wrong thing. It should have restarted
the server instead of exiting.
I will correct the restart-on-crash problem, but since we also need to find
out *where* yours is crashing, we ha
>And yes, when trying to send messages it fails; I can send messages to
>the "admin" account itself.
Damn. I did a fresh install of Debian 10, minimal with no extra software
added, on the exact virtual hardware you described in your last post. I still
can't get it to fail.
Can you
Ok, so you're using stock Debian 10 (64-bit x86, I assume) and the AppImage
declares that it is compatible. You are attaching it to a copy of an existing
database. Local mail works fine, but any attempt to deliver Internet mail
results in a server crash.
Do I have it clear now?
>And although I discovered my mistake and now this VM works as I know it
>should, in my test it still fails to use the "test" of the AppImage, if
>I try to send some mail it just closes the App.
That's progress. :)
So let me see if I understand correctly. If you run the appi
The current version of the AppImage *can* run WebCit on nonstandard ports.
Absolutely.
There is a page at https://www.citadel.org/appimage.html which contains the
downloads and instructions for the AppImage distribution of Citadel. This
is actually the final location; it simply isn't linked from the main part
of the site yet.
If you have a clone of your live system and it was b
>So, it won't help if I add another virtual disk with more storage if I
>can't change the /tmp dump path. Rigth?
Correct. I've removed the misleading guidance from the help text. The
intermediate
dumps must be on /tmp and this cannot be changed.
If you add another virtual disk it
>For the latter, how should I run the database_cleanup to get it to take
>the working path?
database_cleanup always stores its intermediate dumps to /tmp.
I really should change the help text which suggests otherwise.
The results of your tests continue to baffle me. But I am happy to see it,
because if I can fix it for you, that means there are others who would have
had the same result, and we are preventing all those support requests from
happening.
If you have the patience, I would be VERY grateful t
>1. What effect would it have if I run *database_cleanup* with my
>backup+data in the DB?
Yes, I'd try database_cleanup just to see what it does.
Also, I have to ask: is the copy of your database actually in
/usr/local/citadel
or is it somewhere else?
You should also try loggi
This is the developer community only. It's private because we had too many
end users posting poorly triaged bug reports, support requests, and feature
requests in here.
[ http://easyinstall.citadel.org/citadel-1617821022-x64.appimage ]
This version of the AppImage uses the same version of Berkeley DB as the
one used in Easy Install. This means if you've tried the AppImage before
but it choked on an existing database, now is the time to try again!
Ok good. That means the migration tools weren't *completely* broken. We've
come a long way from the days when installing a multiuser Citadel meant
compiling
the sources yourself and carefully putting everything into place. 30 years
ago the objective was to make the administration experience as
>Production is an i5 NUC. I've got 4 of that particular model. Dell
>optiplex 3020 Micro - and two 3040s that have the next gen i5 CPU.Â
Ok, so you're on 64-bit x86. Did you use ctdlmigrate to move from ARM?
I ended up making a lot of changes to that recently when I tried to run it
wit
What are you running your production system on right now?
I just upgraded Uncensored to Citadel 931. I have been experiencing two
problems
lately and I know this will fix at least one of them. One problem was finding
WebCit in an endless loop every morning, and I know that's fixed because I
traced it down to an "optimization" of the HTTP/HTTPS read fun
I'm happy to hear your feedback. One of the reasons I post these internal
updates is to give you (and others who are following the conversation) an
idea of the progress being made. This is how you know what is being worked
on and how close to completion each element has progressed.
Building
Finally ... got a working clone of Uncensored. It took eight hours to copy.
When I do it for real I will have to schedule some actual downtime. :(
I am going through the clone system and so far it appears to be a clean replica
of its source (but running in 64-bit of course). New messag
For anyone following the drama:
No matter what I tried, the SSH connection would eventually choke and die.
Small data, big data, keepalives, whatever I tried it would still die during
the replication. So I removed SSH from the data path. Trying it now using
a direct connection to the so
Geez. It's amazing how badly broken the migration tool was. Thankfully
I am running it against a clone of Uncensored and not the real Uncensored
(btrfs is teh r0x0r) because you folks would be miserable getting on here
:)
Right now I'm dealing with a problem where both ends get a SIGPIP
This sucks and blows. I know people hate it when we deprecate the export
format, but as the commit says, it wasn't working anyway.
I've been procrastinating for years on upgrading Uncensored to 64-bit. It
needs to be done, because I want to use the AppImage on my production system.
This wil
GGGHHH
"ctdlmigrate" has probably been broken for several years, another victim
of untested "optimization". Every time I find these "buffered I/O" routines,
something was broken by it.
In this case, a migration of anything other than the smallest test system
would make the im
If you have to install any dependencies, then the AppImage has failed to live up to its promise.
Hang tight for now. For the past couple of months I've been getting occasional crashes on my production system (this one) and a stack trace shows that it's happening inside of OpenSSL, which is frust
Ever since moving my hosting operation back home, about once a day I would enter my office and find my computer fans screaming like a jet engine, with WebCit on the production system being the culprit. I finally tracked down the problem. Someone "optimized" a function some number of years ago t
I'm less concerned with the hostname stuff because that's largely just configuration and has less to do with the AppImage.
The service crashing on outbound email is more concerning. Are there core files left behind? We need to figure out where the crash is happening. I may have to just try to
https://easyinstall.citadel.org/citadel-1614316873-x64.appimage
I haven't made any changes to Citadel itself in this one. The update in
this version is that the first usable version of an installation procedure
has been added.
You can do "citadel-1614316873-x64.appimage install" and
Interesting. It sounds like we're making progress. I haven't tried it on the public Internet yet, but I have a couple of extra public IP addresses so I'll give that a try next.
Have you considered that the AppImage can update the DB version of my current server?
Can the installation with easy
>You are connected to a Citadel server running Citadel 930.00.
>In order to run this version of WebCit you must also have Citadel 931.00
or newer.
Ok, I believe I have this fixed, but I definitely need to get my builds in
order.
For 64-bit AMD/Intel, try this one:
https:
wtf? How did it build 930 from a version 931 repo?
Time to build a few more test machines :(
https://easyinstall.citadel.org/citadel-1613953314-x64.appimage
https://easyinstall.citadel.org/citadel-1613954272-arm32.appimage
Give 'em a try.
> I'm looking at the complaints in the Citadel Support room from some
>users who are noting the wrong addresses appearing in the to/from
>headers on Internet mail under certain conditions. That looks like
>something that needs to be addressed sooner than later, so I'll be
>looking
Good morning everyone! With my second cup of coffee finished and a 200 foot
driveway full of snow to be plowed, I will post a quick update before heading
outside.
As previously mentioned, there has been a lot of yak shaving going on. It
started with the AppImage development, which forc
Yeah. It isn't glamorous but time spent on stability and consistency is time
well spent.
>Regarding item 4. What will be the mechanism to stop the service using
>the AppImage, just Ctrl-C?
>
>I wonder if it will be with another switch:
># AppImage *stop*
If you're running the AppImage in the foreground, Ctrl-C will still be the
correct way to terminate it. citserve
Hey everyone ... just a quick note to let you all know that the current
objective
is still to make the AppImage work reliably on every platform. It has turned
into a bit of a yak shaving exercise. I am ok with that though, because it
is improving a lot of things under the covers.
1. A
>ctdlvisor: pid=449 exited, status=26880, exitcode=105
Hmm. Exit code 105 means it couldn't open the database.
What version of Berkeley DB is running on the production system you cloned?
If it is an Easy Install system, it's probably 6.2.32? That could be the
problem. Your datab
The correct way to run it would be
./Citadel-86_64.AppImage run -h/usr/local/citadel
Do not specify "/data". That will be added by the program.
All right, here's something new to test.
64-bit AMD/Intel:https://easyinstall.citadel.org/Citadel-x86_64.AppImage
51d5fe3e8274b9b004d846fa1a473b34
32-bit ARM: https://easyinstall.citadel.org/Citadel-armhf.AppImage
1ae5d15065622e85404353370279f0bc
Both files a
Ugh. I opened up a pandora's box with this anti-LFHS thing. Lots of code
to rewrite. Uggghh
>It seems to me that the App is not updated, I download it and it is
>dated *January 15th*. And if I run it I get errors that had already been
>solved.
Weird. I must have only done the ARM version. But I'm not going to waste
your time because I found some problems related to the wh
>I'm going to restore a snapshot of my VM so I can try again; when I try
>to log in with http; and the "citadel" process just crashes.
Yes, definitely use a new snapshot. If you ran the previous version of the
appimage, you can pretty much count on any database it touched being corrupte
Just use HTTP for now. I will work out the certificate stuff as part of the installation process. Again, right now I just want to know that it runs on your system. Yesterday I figured out why it was corrupting the database: when you pressed Ctrl-C it somehow got two copies of citserver running
AppImage development is teaching me something about the rest of the server:
we spend way too much time "cleaning up" when the server is exiting. I am
now removing thousands of lines of "exit cleanup hook" code that doesn't really
need to be there. Kudos to whoever wrote it, they had the inten
No, keep using whatever you were using. We want the AppImage to be tested on as many different systems as possible. If everyone is using what I'm using, that misses the point.
I'm working on this today and I'm going to try to figure out why WebCit keeps going catatonic on us (insert your favori
That's kind of odd. Ctrl-C works for me on both x64 and ARM.
I'm concerned about the "buffer overflow detected" error. That's a new one
for me. What operating system are you trying it on?
> * Will it be possible to recover space from the DB by compressing
>what is deleted? (because I see that the DB is still +90GB)
In the version of Citadel you are running in the AppImage, you should now
be able to set the hidden "c_shrink_db_files" configuration option, which
will run a
Excellent. Happy to hear that it works.
For better performance, do I need to install a local DNS like BIND?
That's up to you and not related to Citadel. If you want a caching DNS on your local host, I would recommend dnsmasq over BIND, but again, Citadel doesn't care.
If my current ser
>Citadel Server found BOTH legacy and new configurations present.
>Exiting to prevent data corruption.
There's your problem. If you ran Citadel server without citadel.config and
citadel.control present, it would have generated new configurations. Now
it sees two conflicting configuratio
Definitely move citadel.config, citadel.control, and refcount_adjustments.dat down into /usr/local/citadel/ (not in data/). We want the database directory to contain only the database.
At the VERY least, you're going to need to find citadel.config and
citadel.control
files, and copy them to /usr/local/citadel/ along with your database in
/usr/local/citadel/data/
https://www.citadel.org/file_layout.html might give some hints as to where
to find the rest of the files, bu
And I can't connect to webcit: This program was unable to connect or stay connected to the Citadel server. Please report this problem to your system administrator. I wonder if it won't work for me to copy the DBs from my current version: Citadel 8.17 with WebCit 8.17, server build 8.17? (in
Unfortunately a Mac port has been elusive, but we're gradually working it
out. We have at least one person who has been interested in following the
progress on that. It should be clear, however, that AppImage doesn't run
on Mac ... only Linux.
The final package will probably have syntax something like this...
citadel.AppImage run [-h dir] [-p port] [-s port]-h : Operate with data directory dir-p : Run the HTTP server on port port-s : Run the HTTPS server on port port
citadel.AppImage stop(stops a running instance of the Citadel system)
Of course. No hurry. While the lot of you are testing for compatibility,
I will be writing all of the installation and maintenance scripts around it.
My hope is that this will work so well that it becomes the primary way people
install and run Citadel. "Even Easier Install" or something like th
What you experienced is exactly what it is supposed to do at this stage, no
more, no less. There is not yet an option to "install" it, nor is there an
option to go with anything other than the default configuration and location.
When you run the appimage, the entire system starts up; when you ctr
All righty then! It's time for round 2 of testing the Citadel appimage.
64-bit x86: https://easyinstall.citadel.org/Citadel-x86_64.AppImage
32-bit ARM: https://easyinstall.citadel.org/Citadel-armhf.AppImage
(A note on the ARM build: the Raspberry Pi OS is 32-bit, even on 64-bi
Yeah. Hang tight, I've already figured out the webcit problem in the current
build, and I just need to make sure I didn't break Easy Install or manual
builds with it. I probably broke the Debian build, but as explained in a
previous post I can't commit to supporting that style of installation any
I get this message when I try to get in: http://192.168.16.33/ didn't find Template [login] 5 5
Great. Don't worry about webcit barfing on the template yet. I am getting that too, and am in the process of fixing it.
For the time being, I am satisfied to know that the AppImage is binary-compa
After a long amount of contemplation, I've decided to remove support for
LHFS in the Citadel System. This effectively ends the ability to build .deb
and .rpm packages that can be included in Linux distributions. I have resisted
this for years, but it is creating more problems than it solves
Are you still running your production system on a Raspberry Pi? Hopefully
I will have an ARM image ready by early next week. I think you should do
your testing on a Pi 400 :)
The new AppImage for Citadel is functionally complete. WE NEED TESTERS!
Obviously, don't try this on a production system. I would like as many people
as possible to test this image, to see if the promise of a universal single-file
binary distribution has been met. Right now it only r
More progress has been made on the AppImage. It now contains a small supervisor
program that will keep the various server processes up and running, restarting
them if they crash for whatever reason, etc. It shouldn't take long to finish
this.
I am excited about it. If this works as expected
The development team has always operated in that mode -- people come and go,
some contribute amazing things, some make messes that have to be cleaned up,
but it's always good to have some fresh ideas. There are occasionally stretches
where I'm the only one writing any code, but we must remember th
mo: that's pretty much our target audience at this point. I've stopped thinking
about the large enterprise market because those people are stupid, and stupid
people prefer Exchange (and Sh*point, and now Teams). A lot of what we were
building in the past revolved around the idea that a really lar
Tonight I have been playing around with AppImage and it's looking promising
so far.
With just an hour or two of tinkering I was able to build an AppImage
containing
the Citadel Server and all of its dependencies, wrapped up all nice and snug
in a single binary executable. The really neat
Well, I wasn't specifically looking for a donation, but if you're willing,
I'd be pleased to accept. It would basically become a build test host for
Citadel on ARM.
This is an idea that may have potential. Tell me what y'all think of it.
One thing we know is that even with Easy Install, some people still have trouble getting Citadel built and running. What if we built the entire Citadel suite as an AppImage [https://appimage.org/] ? These are designed to
All right, finally some sanity.
The weirdness with the webcit upgrade turned out to be from a pair of field
lists in two different header files that depended on being the same size and
in the same order, and the person who did that failed to include any comments
pointing that out.
It'
*sigh*
Still mitigating the effects of making the "local name" and "name + email
address" address syntaxes more standardized.
I really, really, really hate legacy webcit. Really, really looking forward
to the day webcit-ng is feature complete and we can ditch the old bucket of
bits.
So let's hit up some of the security issue reports. I'd like to start with this one:
Valid Request:
GET /ajax_login_username_password?name=admin HTTP/1.1Host: 192.168.1.239:8080
Valid response:
540 Wrong password.
Invalid Request:
GET /ajax_login_username_password?name=notregistereduser HTTP/1
As for why this is taking so long.
WebCit is broken. Really, really broken. Some time ago we embarked upon
an effort to get all of the HTML out of the WebCit C code and put it into
templates. And we did. The problem is that the template syntax is so horrible
that it's barely readable,
Technical discussion of the last few commits, for anyone interested:
In "classic" Citadel there were several different address formats:
1.Display name
2.Display name @ node
3.Display name
The second format is clearly obsolete now that there is no more
>Author: root
>Date: Wed Dec 2 09:36:26 2020 -0500
>
>input type="date" here will allow datepicker.js to be eliminated
>and should fall back to type="text" in the few remaining browsers
>that don't support it.
I'm assuming this is Warren ... could you set your c
I've created a private wiki called "Citadel Issue Tracker". It is a hidden
room. In the past we've had a terrible record with bug trackers because they
tend to fill up with badly reported issues, badly tested issues, and feature
requests. So I'm making the audience limited for this one becau
The above commit should be the final completion of the new libsieve-free
inbox rules handler.
It probably needs a bit of testing, if anyone has the ability to do so.
Once this is all buttoned up we can publish a new version of Citadel, which
will also include the fixes to the RSS read
This is a quick note about Citadel Server internals that records something which is on my mind today. It can be digested as a point of interest or converted to technical documentation if anyone is interested. It revolves around the database table called S_USETABLE, whose format is:
struct UseTa
> 2020-11-22 00:42 from rss
>Subject: buntu 20.04
>In this article, weâll discuss how to install Flask on Ubuntu
>20.04 inside a Python virtual environment.
>
> https://linuxize.com/post/how-to-install-flask-on-ubuntu-20-04/
>
See that? The subject is
>I don't have it up here in front of me, but there is the section that if
one
>is empty, the other is populated. That may have dredged up things we never
>knew existed.
Right. I believe I have fixed that.
Somewhere along the line, another developer made struct CtdlMessage a bit
>There is all kinds of junk in those subject lines.. even CDATA. Also
>"Author/Creator". I think the Author/Creator code should be revisited...
Are you making this observation from an audit and/or test of the latest commit,
or from what is on Uncensored? I'm not *running* the fixed c
Regarding the proper RFC 2047 encoding of Subject and other lines in RSS
feeds...
I saw the "fixed_01" version of serv_rssclient.c and I don't understand why
it makes calls to html_to_ascii. HTML is not allowed in subject fields,
is it?
I fixed the problem (or at least I think I did
You have write access to the repository. If the fix is well tested, go ahead
and put it in.
>What mainly caught my attention is the inbox filter engine. I'm not sure
>(perhaps I should read back some.. ) exactly what you're referring to.
libSieve is old and unmaintained, and most people just want to write inbox
rules using the rule editor anyway, so we're abandoning Sieve an
It's been said that a "POP Toaster" (a standalone mail server that does, at
minimum, POP and SMTP) is an application that any decent hacker can throw
together in a weekend. I found this one called "XMail" [http://xmailserver.org]
that was started in 1999 and apparently abandoned in 2010. It doesn
You know what ... now that I'm more awake, I remember more clearly :)
What I was thinking about was actually called "unstripped" not "unpatched".
There was a period of time during which the build produced binaries both
with and without the debugging symbols intact.
>Thought to make the comment.. big difference in the size of the binaries..
>"unpatched" was a recent (like, week or so) easyinstall..
What's the date on the unpatched version? I thought we got rid of that a
long time ago.
There was some sort of build/diagnostic option that I neve
Ok never mind about the upload, I viewed your message in WebCit and the change
that needed to be made appeared clearly.
Before I commit this, by what name do you want to be credited in the changelog?
>Okay it's fixed. I'm very happy about this. Lots of work to find, butÂ
I
>asked God to help me find it, and He did. :)
Nice! Also I didn't know you were a believer, that's also very cool.
We should definitely apply your patch to the main tree immediately, but could
you upload it
Hmm. This stuff has been reworked a few times since I last worked on it,
but it looks like you would be interested in following the following code
paths:
CtdlMakeMessage() in msgbase.c (code to read in messages ... this is called
by the SMTP server and others)
convert_internet_message()
Believe me, I get it. An exodus from the big sites is exactly what I want.
Facebook and Twitter are cancer on the Internet and must be eradicated.
Recently I've begun thinking that the failure of our largest cities is the
exact same thing happening in meatspace, and what we really want is to be
It's a good question, and one for which I have an answer in my mental roadmap
for the Citadel system.
First let's address the idea of a "messaging monopoly". During the dialup
era, the potential audience of a BBS was limited to those who could reach
the site with a local call, and to a lesser
At this time the revisions to the wire protocol, configuration storage, and
user interface have been completed for the transition from Sieve to a
rules-based
filter.
Now I just have to write the filter itself :)
FINALLY.
After a lot of work and even more procrastination, the new
http://www.citadel.org
site has been completed and published.
This is kind of a big deal because I've been trying to have the self-discipline
to finish the site before I go back to writing any more code. Documentat
After some consideration I think I agree. Hopefully the number of people
using custom Sieve scripts is very small.
When someone uses the rules-based filter available in WebCit, here's what's
actually happening:
1. The rules are converted to a Sieve script
2. The rules are then encoded
I am having a first look at the conversation in Citadel Support (will look
closer at it later) and noticing that libSieve is involved in someone's build
troubles.
libSieve [https://github.com/sodabrew/libsieve] hasn't been updated in 8
years. It is a small library. I am starting to think
201 - 300 of 2224 matches
Mail list logo
