Re: S3 Object Storage
Hello Ryan, On 2020-04-06 00:45, Ryan Blenis wrote: Thank you as always for your quick and in-depth response. I certainly understand the delicate balance between FOSS and enterprise paywalls and feature-sets, and I thank you for even having an open source option to begin with! I'm just planning a migration to a new server (for when my current OS is EOL) and wanted to move the hosting to one with better support that doesn't over-provision the hosts resources as I've seen on the current host, but HDD space isn't anywhere near as inexpensive as it is at the current company. I was just thinking I could have the best of both worlds with S3 storage integrated. Many of the FOSS/enterprise paywalls in other projects are based on LDAP integration, multi-tenancy, pre-packages binaries, support levels, etc. that many business need, whereas object storage vs block storage felt like more of a separate paywall in terms of cloud providers, so I just figured I'd ask if it was coming to the FOSS version. Also, I it's ok to ask, and I always appreciate the feedback from piler users. Btw. can you share the names of these 2 providers? I'm curious about their pricing. And perhaps the piler community may suggest some better alternatives both in terms of hw performance and storage pricing. like to be able to modify the code if a customization can be made, which I assume would not be the case with enterprise, correct? Not sure if you fix the C source code before compiling. If not, then it's probably not a big deal that you can't modify the enterprise edition binaries. However, the gui part is still the usual php based stuff, it's not obfuscated, so you can fix it if you really want to do that. It's your archive after all. Custom css is supported out of the box. Also, if I try the enterprise on the new host, is there a path to downgrade to the FOSS version? Or would starting back from scratch be the only option? Unfortunately no. The enterprise edition handles attachments in a different way: it doesn't need the attachment table to retrieve the message with its attachments. However, it's incompatible with the open source version. So if you decide later that it's not what you want then you should export all emails, install the open source edition, and import those emails. Though, I'm confident that you'll like it, it's not inferior to the open source edition. Janos SUTO On Sun, Apr 5, 2020 at 7:16 AM wrote: Hello Ryan, On 2020-04-05 10:55, Ryan Blenis wrote: I see the enterprise version you offer it says one of the features piler has is S3 object storage, which looks like it was originally requested on the mailing list in 2018: https://www.mail-archive.com/piler-user@list.acts.hu/msg01335.html Does this mean that it is fully implemented now? And if so, will we yes, it's fully implemented by now. ever see this in the open source version? Given the lower prices on S3 compatible storage versus block storage and the unchanging nature of the files, this would be great to have as an option that can reduce hosting costs from what is now being used for [comparatively] expensive block storage. well, I've been thinking about it for a while, and still not decided, it's a difficult business decision, so let me share my current position on the matter. TL;DR: not in the near future. I'm really proud that the open source edition has gained a pretty nice traction, and I decided to somehow monetize its popularity. So I've forked piler and built an enterprise edition along with some commercial services. To make the enterprise edition successful, I must offer some features that are worth to pay for that is not available for free in the open source edition. I believe that the S3 storage support you asked for is such an attractive feature, so I'll keep it for the enterprise edition only, not sure how long. However, I'm well aware of that the S3 costs are significantly lower than block storage costs at cloud hosting companies. I encourage you to try the enterprise edition, and see if it works for you. If so, and you like it, then perhaps we could figure out something viable to make the transition from open source to enterprise. I'd like to emphasize that I don't want to force you in this direction, and it's totally ok to decline. Let me know what you (and other piler users as well) think about the matter. Janos
Re: S3 Object Storage
Hi Janos, My preferred provider is Linode, I've had a couple VM's with them for years and they've always been top-notch. I won't mention the other provider (I'm not one to bash) currently hosting my Piler instance that I see service drops on almost every morning, but I will say that (failing to have S3 support) I'd be looking to migrate from them to RamNode as they appear to have decent reviews and a "spinning disk" option with a good amount of space (comparable to that of the current host). So if you/anyone has any praise or warnings of RamNode hosting Piler, I'm all ears. I have not yet had to dabble in the C code for anything, but am familiar with C/C++ and a few other various languages, so I always like to have the option. It's good to know the web UI is still open in the enterprise version. As for trying enterprise at this time, I suppose I'll try the RamNode route first with spinning disk block storage. If the provider isn't as stable as I'd like I may have to reconsider, but for now it will be a matter of cost of licensing Piler purely for the S3 capability versus minimizing my hosting provider options to those with larger storage options. If I require a more stable host and can't find any with inexpensive storage, it just comes out to a cost calculation between block storage cost in a provider versus Piler enterprise costs for S3 compatibility... unless of course you come out with a discounted S3 "add-on license" for just that feature that makes the math a no-brainer. I've been using Piler for so many years personally, I should probably just start pushing clients toward it, but I've had some issues with it missing mails due to service outages, failing to load the UI unless the service is restarted (after running for several days), failing to clear the sph_index queue on a semi-regular basis, and a few other quirks along the way that made me pause before offering it as an actual service in a customer-facing manner. I've automated most of that fixing itself with Zabbix monitoring, but some issues remain. Honestly though, I've attributed most if not all of that to the current VPS host. If the new version I install on a new host sees all those issues go away, and I'm comfortable having potential clients log into the interface and know something random isn't going to happen to them, the easy call is to push clients toward it and get an enterprise license anyway. So here's hoping that's the case! I'd love to get rid of the proprietary options I've got some clients on and move everyone to Piler if possible. Thanks again Janos! On Mon, Apr 6, 2020 at 3:11 AM wrote: > > Hello Ryan, > > On 2020-04-06 00:45, Ryan Blenis wrote: > > > > Thank you as always for your quick and in-depth response. I certainly > > understand the delicate balance between FOSS and enterprise paywalls > > and feature-sets, and I thank you for even having an open source > > option to begin with! I'm just planning a migration to a new server > > (for when my current OS is EOL) and wanted to move the hosting to one > > with better support that doesn't over-provision the hosts resources as > > I've seen on the current host, but HDD space isn't anywhere near as > > inexpensive as it is at the current company. I was just thinking I > > could have the best of both worlds with S3 storage integrated. Many of > > the FOSS/enterprise paywalls in other projects are based on LDAP > > integration, multi-tenancy, pre-packages binaries, support levels, > > etc. that many business need, whereas object storage vs block storage > > felt like more of a separate paywall in terms of cloud providers, so I > > just figured I'd ask if it was coming to the FOSS version. Also, I > > it's ok to ask, and I always appreciate the feedback from piler users. > Btw. can you share the names of these 2 providers? I'm curious about > their pricing. And perhaps the piler community may suggest some better > alternatives both in terms of hw performance and storage pricing. > > > > like to be able to modify the code if a customization can be made, > > which I assume would not be the case with enterprise, correct? > > Not sure if you fix the C source code before compiling. If not, then > it's probably not a big deal that you can't modify the enterprise > edition > binaries. However, the gui part is still the usual php based stuff, > it's not obfuscated, so you can fix it if you really want to do that. > It's your archive after all. Custom css is supported out of the box. > > > Also, if I try the enterprise on the new host, is there a path to > > downgrade to the FOSS version? Or would starting back from scratch be > > the only option? > > Unfortunately no. The enterprise edition handles attachments in a > different > way: it doesn't need the attachment table to retrieve the message with > its > attachments. However, it's incompatible with the open source version. > > So if you decide later that it's not what you want then you should > export > all emails, install the op
Re: S3 Object Storage
Hello Koby, On 2020-04-06 07:07, Koby Peleg Hen wrote: Hello all , I am currently on this , Please Janos , make your decision. Since I have looking at your project for very long time , And my client begin to ask for. It is very simple to implenent , because there is simple convertor from regular file system to S3. If you decided not to , please let us all know. the answer is no for now. I've googled the topic, and found s3fs (perhaps you referred to the same project). Found a few articles that might come handy: https://cloud.netapp.com/blog/amazon-s3-as-a-file-system https://medium.com/swlh/using-s3-just-like-a-local-file-system-in-python-497737783f11 I encourage you and other piler users as well to test it. Perhaps it's a good enough solution, and works with piler properly. If so, then I'll update the docs with your findings. In this case I will need to implement it by my self by the near future. I have a lot of respect to your project and your time and the willing to get pay for it. But if some one whould like to pay , is for the support not for feature. Please take a look on the proxmox project. I just did. And looked at nginx plus as well. Some projects monetize support, others the features as well. By my self , and my clients , I do not mean to take off without S3. I'd say give a shot for s3fs, and see how it works. I'm not against it (or any usable solution) by any means. Janos SUTO