Hi John; On Sun, Nov 1, 2015 at 7:13 PM, John Locke <[email protected]> wrote:
> Hi, > > Love to hear the directions and priorities. Overall, I'm completely > onboard, and support these. Couple comments: > > On 10/31/2015 11:50 PM, Chris Travers wrote: > > >> There are a few things architecturally however that I would hope to >>> accomplish with 1.6: >>> >>> 1. From 1.2 through 1.4, ease of installation has not been a priority >>> and has degraded We need to reverse that. Ease of installation and >>> management needs to be a top priority. I think we have the back-end stuff >>> down, so the question is front-end and administrator tooling. >>> >> >> I would say this is particularly true: we don't have a web tool which >> points out which libraries or other dependencies are missing from an >> installation. Tools like Zabbix have that, allowing an admin to quickly >> address the issue(s) and assess the level of support for any of the >> optional features/dependencies. This could seriously improve our "first >> user experience". >> >> >> >> Well, I do think that using Docker can alleviate a lot of this pain, in 2 > ways: > > 1. Provide a "ready-to-run" installation provisioned in two steps (run a > Postgres container, run an LSMB container). > > 2. Provide a supported installation recipe for anyone who wants to run it > themselves (the Dockerfile). We could support multiple flavors of this: > install from git (what the current dockerfile does), install from tarball, > install from package. And of course, some steps for creating a development > environment (e.g. use Docker to provide all dependencies but mount a git > tree into the container that you can develop on directly). > > There are some small blockers/disadvantages here, but this is close to > being ready for use, and I've been using a Dockerized install in production > for many tasks. The current downsides/blockers: > > - Size. Current images are > 1.5GB, largely due to LaTeX and other > dependencies. > You can probably cut down on the size of the LaTeX install. The problem is that the TexLive bundles that include everything we use are large, but they include tons of things we don't and lots of documentation. I don't know if you are installing the docs but the basic LaTeX install with the specific cls and sty files we use wouldnt be very big. > - Issues #854, #917 -- templates cannot be easily managed in the > filesystem in Docker, so until all templates can be loaded into the > database, this is impractical for casual users. > - Server-side printing -- this might be something relatively easy to solve > by setting up some print variables and mounting a print device inside the > container, have not addressed this yet. > > > So we currently use the regular CGI for printing checks and sending > invoices, but have migrated most other activities into Docker, with the > much faster Starman configuration (2 instances sharing the same database). > And for new users, this is a really fast way to get a working install up > and running. > Interesting. > > > > > On a technical side, I would like to switch to external tooling for >>> administration and management. This means one gets the admin tools, and >>> can install and manage LedgerSMB instances from there. It also means these >>> tools need to play really well together. >>> >> >> I'm affraid this sounds pretty vague from just this sentence. I'm affraid >> people will have a hard time judging how this will impact them. Will we >> still deliver these tools in such a way that installation will be easy? >> Will these be part of the source distro so packagers for distributions >> automatically get the right sources to package and distribute? Could you >> elaborate a bit more on your plans in that respect? >> >> > Other than that, if the tools come with the same source tarball and are a >> huge improvement over what we have now, I think we only can cheer to that >> :-) >> > > I was actually thinking of a couple different approaches we could take > here. > > 1. Install the admin and installation tools. Run them to check your > system, download, and install latest versions of whatever major branch you > want (starting with 1.6). Such could also update a system and could be run > with different permissions than the main web server. In other words, you: > a. download a tarball. > b. Run a program. It checks your system. If it can start a web > server on a higher port it does so. Otherewise it gives clear feedback > about what you are missing. > c. Log into the web interface for more system diagnostics, > installation and management help. > > Cool idea. But is this the best use of our time? > > With the rise of Docker, I think a lot of this is becoming largely > unnecessary. We can create a single supported environment that can run > anywhere, even on Windows, and not have to support all the possible > variations of environments -- we should be focusing our efforts on the > financial system rewrite, not admin tools to fix a bunch of different > environments. > > If somebody really wants a bare-bones install in their specific > environment and not run in a container, that sounds like a perfect > opportunity for paid consultants to step in -- for the core team, this > seems like a huge distraction. > Well, the sorts of things that go into the Setup.pl are really not going away any time soon (or ever) and the tooling there is less than ideal. I have a partial replacement there. > > > 2. We could bundle all our tools together. In this case you still run > the same program which checks your system, sees whether it can run a web > server and if not gives information as to why not. Such a program would > then typically not update a system but could change file permissions after > running and setting things up. > > In both cases, the installer checks external prerequisites, CPAN > dependencies, and can either solve or advise on how to solve missing > dependencies. I would like LedgerSMB 1.6 to be easier to install than > SQL-Ledger 2.6 was. Mostly this is hand-holding, providing clear feedback > on how to resolve problems, and a nice interface for showing the > information. > > I would argue that we should skip all the dependency-checking, etc, and > point people either to packages for Debian/RHEL/etc, or the Docker images. > For RHEL users, there are currently a lot of extra dependencies to track down that are not in the standard repos, and if you are installing on a Linux server, you really need to know how to set up PostgreSQL etc. So having a nice installer would help out a lot. If we want to go this route, we really need our own yum/dnf repo. I suppose we could host one at Efficito. > > Where we do need to spend time hand-holding is once the system is up and > running -- setting up the Chart of Accounts, providing guidance on user > permissions, how to set up bulk imports, etc. Nice interfaces for showing > best accounting practices, templating guidance, etc. > Agreed on this. This also means db creation and management, etc. In other words ,the area that App::LedgerSMB::Admin (and ::Web) have focused on regarding a next generation cli and web interface. These are the external tools I would like to integrate. It does mean an additional couple dependencies which means getting dependency tracking is a little more important. > > > > I also want to make sure we have both command line and web tools. > Command line tools give experienced administrators a productivity edge here > (because it is easier to quickly send complex information to the program), > but web tools give new users an ability to get up and running with less > immediate knowledge. > > > By "command line" tools, do you mean something more than what's already > there -- the ability to run any of the .pl scripts directly and pass in > info using a Bash script, for example? > The immediate ones that come to my mind are: 1) Create a database with a specific coa, etc. 2) Add a user to a database 3) Update a database within a stable branch 4) Upgrade a database from one stable branch to another. These would help out quite a bit where multiple dbs are managed in the same organization. Docker or no, having a good scriptble admin interface makes things a lot easier. Now, for 1 and 3, I already have this fairly well working. For 2 and 4, there are some complications. But they come in really handy in a number of cases, not only hosting itself but holding companies, subsidiaries, etc. where you have an IT team which can use tools like this to automate the update process. It also helps where the web server does not have superuser access to the db for security reasons. And I have run into enough cases where this was necessary to know that updating and creating dbs from the command line is very helpful. > > I'm not sure command line/shell tools are the real need -- I think the > critical thing is to get an API in place, bit by bit. We can certainly wrap > those with a shell tool, but I think the real need is for a solid API. > I totally agree about the need for a solid API. For the admin tools, the approach I took was: 1) An admin library 2) CLI tools that use it 3) A dancer-based web interface. My proposal is that we extend these tools nd replace our existing management tools with them. I don't think CLIs will work for things like invoices. It is hard enough to come up with good user interfaces for adding users on the command line. But for sysadmin tasks they are very important. But if someone needs them, if we have good solid Perl API's we can build cli's too. But in terms of an API, going from a solid Perl API to a web service with Dancer is pretty trivial. Going from a good DB API to a Perl API is also pretty trivial with PGObject. So if we have good Dancer integration and good DB API's then we are very close to have solid, stable API's for third parties both in cases where you want to go through a web service and in cases where you might not. The previous blockers involving passing a gl transaction into or out of the db as a single data structure have now been solved, so.... > > The rest of the mail goes into specifics, which all seem appropriate to > me. I just wanted to make the point that an API can help us clean up the > legacy code. If we start developing a solid API, module by module, and > using that to interact with the existing financial code, then it becomes > far easier to swap out the implementation when we get there. I know this is > easier said than done, given the spaghetti logic in there ;-) > > Perhaps the API can be done as a 2-step approach? I'm thinking we already > discussed tagging the API with a version. What if we start a v2 right now > -- v1 being essentially a wrapper around the existing POST variables that > all the forms currently expect, and v2 being what we think it should look > like, where this varies? Then for each module, we could go through the > current code, and replace each variable in the module that comes from the > browser/form post with something that we set in the API handler for that > module -- and can then create both v1 and v2 at the same time. Once we have > a stable v2 API across the board, then we have a great framework for unit > testing in place and can far more reasonably rip out and rewrite the core > logic. > I am not convinced it is practical to go to v2 directly, at least not in a single release in LSMB or in a phased direction there. I think anything hitting the old db structures is going to be v0 -- subject to change without notice. The current db structure really is confining in not very nice ways. To come up with something we like we are going to have to redesign it and that alone is a large task. > I'm seeing the API as functional scaffolding to help with the financial > rewrite, as well as being an end in itself... > Agreed entirely. > > Cheers, > John Locke > http://www.freelock.com > > > > ------------------------------------------------------------------------------ > > _______________________________________________ > Ledger-smb-devel mailing list > [email protected] > https://lists.sourceforge.net/lists/listinfo/ledger-smb-devel > > -- Best Wishes, Chris Travers Efficito: Hosted Accounting and ERP. Robust and Flexible. No vendor lock-in. http://www.efficito.com/learn_more
------------------------------------------------------------------------------
_______________________________________________ Ledger-smb-devel mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/ledger-smb-devel
