Re: [Assp-test] fixes in assp 2.4.4 build 15067
Colin, >If I understand more of how experimental this is >and the next step is to HA the database At this time the code is very experimental and very special. It aims to fix SMTP performance problems for an ISP, which holds around 20.000 domains. The concept of the central RDB (for HMM and Bayesian) backend is not fast enough to process several hundred thousands or million mails a day. If 100.000 mails have to be processed with HMM and/or Bayesian in a day, this will lead in to 6.000.000 - 60.000.000 SQL queries a day (only for HMM). What DB engine (cluster) is able to do this? And this is only the average calculation - what about the peaks? The code is currently specialized for the environment of this single ISP and is not generic enough to go public. There are currently no changes made to enhance the implementation of other features, like blockreporting or anything else. Thomas Von:Colin Waring An: ASSP development mailing list Datum: 08.03.2015 15:18 Betreff:Re: [Assp-test] fixes in assp 2.4.4 build 15067 Hi Thomas, I'd be very interested to know more details on the ultimate aim with the ISP option. I support the idea of subscription for the higher end as it will help create funding for you past donations. Is the aim of the addition to add support for extended scalability or do you have ideas for the future to make additional features available? If you remember we exchanged emails a while back about some of the features that I could see benefiting a larger setup and we are looking into how to implement things at the moment. I've already implemented clustered file systems and the next step is to HA the database. The biggest concern for me in scaling up is the block reports being generated on each server individually. If I understand more of how experimental this is and what could go wrong then I may be able to help with testibg. All the best, Colin Waring On 8 Mar 2015 12:39, Thomas Eckardt wrote: Hi all, fixed in assp 2.4.4 build 15067: - on some windows systems 'Win32::Unicode' was detected as unavailable, even it was correctly installed - the alpha index was not working in build 15059 - HMM was not working, if 'spamdb' was set to a plain file, placed in a subfolder like: db/spamdb added: - This build contains experimental code to setup assp in very large ISP environments, with a very high workload caused by HMM, Bayesian and DNS. Such a setup requires an enormous and expensive amount of hardware resources, a very high knowledge in system design and OS scripting. minimum requirements: - assp: 64Bit OS, all SSD, 16GB RAM, 8 CPU cores, 64Bit Perl (multiple larger systems expected) - external high available enterprise database server - high available and very fast DNS-servers This ISP setup option is subject to become a payed licensed feature. Thomas DISCLAIMER: *** This email and any files transmitted with it may be confidential, legally privileged and protected in law and are intended solely for the use of the individual to whom it is addressed. This email was multiple times scanned for viruses. There should be no known virus in this email! *** -- Dive into the World of Parallel Programming The Go Parallel Website, sponsored by Intel and developed in partnership with Slashdot Media, is your hub for all things parallel software development, from weekly thought leadership blogs to news, videos, case studies, tutorials and more. Take a look and join the conversation now. http://goparallel.sourceforge.net/ ___ Assp-test mailing list Assp-test@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/assp-test -- Dive into the World of Parallel Programming The Go Parallel Website, sponsored by Intel and developed in partnership with Slashdot Media, is your hub for all things parallel software development, from weekly thought leadership blogs to news, videos, case studies, tutorials and more. Take a look and join the conversation now. http://goparallel.sourceforge.net/ ___ Assp-test mailing list Assp-test@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/assp-test DISCLAIMER: *** This email and any files transmitted with it may be confidential, legally privileged and protected in law and are intended solely for the use of the individual to whom it is addressed. This email was multiple times scanned for viruses. There should be no known virus in this email! *** ---
Re: [Assp-test] fixes in assp 2.4.4 build 15067
>does this mean that if one upgrades ASSP it will not work anymore due to system constraints ? No, this feature requires a special setup. As long as this setup is not done, assp works like before. Thomas Von:grayhat An: assp-test@lists.sourceforge.net Datum: 08.03.2015 14:44 Betreff:Re: [Assp-test] fixes in assp 2.4.4 build 15067 It was Sun, 8 Mar 2015 13:38:51 +0100 when Thomas Eckardt wrote: > Such a setup requires an enormous and expensive amount of hardware > resources, a very high knowledge in does this mean that if one upgrades ASSP it will not work anymore due to system constraints ? -- Dive into the World of Parallel Programming The Go Parallel Website, sponsored by Intel and developed in partnership with Slashdot Media, is your hub for all things parallel software development, from weekly thought leadership blogs to news, videos, case studies, tutorials and more. Take a look and join the conversation now. http://goparallel.sourceforge.net/ ___ Assp-test mailing list Assp-test@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/assp-test DISCLAIMER: *** This email and any files transmitted with it may be confidential, legally privileged and protected in law and are intended solely for the use of the individual to whom it is addressed. This email was multiple times scanned for viruses. There should be no known virus in this email! *** -- Dive into the World of Parallel Programming The Go Parallel Website, sponsored by Intel and developed in partnership with Slashdot Media, is your hub for all things parallel software development, from weekly thought leadership blogs to news, videos, case studies, tutorials and more. Take a look and join the conversation now. http://goparallel.sourceforge.net/ ___ Assp-test mailing list Assp-test@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/assp-test
Re: [Assp-test] fixes in assp 2.4.4 build 15067
Hi Thomas, I'd be very interested to know more details on the ultimate aim with the ISP option. I support the idea of subscription for the higher end as it will help create funding for you past donations. Is the aim of the addition to add support for extended scalability or do you have ideas for the future to make additional features available? If you remember we exchanged emails a while back about some of the features that I could see benefiting a larger setup and we are looking into how to implement things at the moment. I've already implemented clustered file systems and the next step is to HA the database. The biggest concern for me in scaling up is the block reports being generated on each server individually. If I understand more of how experimental this is and what could go wrong then I may be able to help with testibg. All the best, Colin Waring On 8 Mar 2015 12:39, Thomas Eckardt wrote: Hi all, fixed in assp 2.4.4 build 15067: - on some windows systems 'Win32::Unicode' was detected as unavailable, even it was correctly installed - the alpha index was not working in build 15059 - HMM was not working, if 'spamdb' was set to a plain file, placed in a subfolder like: db/spamdb added: - This build contains experimental code to setup assp in very large ISP environments, with a very high workload caused by HMM, Bayesian and DNS. Such a setup requires an enormous and expensive amount of hardware resources, a very high knowledge in system design and OS scripting. minimum requirements: - assp: 64Bit OS, all SSD, 16GB RAM, 8 CPU cores, 64Bit Perl (multiple larger systems expected) - external high available enterprise database server - high available and very fast DNS-servers This ISP setup option is subject to become a payed licensed feature. Thomas DISCLAIMER: *** This email and any files transmitted with it may be confidential, legally privileged and protected in law and are intended solely for the use of the individual to whom it is addressed. This email was multiple times scanned for viruses. There should be no known virus in this email! *** -- Dive into the World of Parallel Programming The Go Parallel Website, sponsored by Intel and developed in partnership with Slashdot Media, is your hub for all things parallel software development, from weekly thought leadership blogs to news, videos, case studies, tutorials and more. Take a look and join the conversation now. http://goparallel.sourceforge.net/ ___ Assp-test mailing list Assp-test@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/assp-test -- Dive into the World of Parallel Programming The Go Parallel Website, sponsored by Intel and developed in partnership with Slashdot Media, is your hub for all things parallel software development, from weekly thought leadership blogs to news, videos, case studies, tutorials and more. Take a look and join the conversation now. http://goparallel.sourceforge.net/ ___ Assp-test mailing list Assp-test@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/assp-test
Re: [Assp-test] fixes in assp 2.4.4 build 15067
It was Sun, 8 Mar 2015 13:38:51 +0100 when Thomas Eckardt wrote: > Such a setup requires an enormous and expensive amount of hardware > resources, a very high knowledge in does this mean that if one upgrades ASSP it will not work anymore due to system constraints ? -- Dive into the World of Parallel Programming The Go Parallel Website, sponsored by Intel and developed in partnership with Slashdot Media, is your hub for all things parallel software development, from weekly thought leadership blogs to news, videos, case studies, tutorials and more. Take a look and join the conversation now. http://goparallel.sourceforge.net/ ___ Assp-test mailing list Assp-test@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/assp-test
[Assp-test] fixes in assp 2.4.4 build 15067
Hi all, fixed in assp 2.4.4 build 15067: - on some windows systems 'Win32::Unicode' was detected as unavailable, even it was correctly installed - the alpha index was not working in build 15059 - HMM was not working, if 'spamdb' was set to a plain file, placed in a subfolder like: db/spamdb added: - This build contains experimental code to setup assp in very large ISP environments, with a very high workload caused by HMM, Bayesian and DNS. Such a setup requires an enormous and expensive amount of hardware resources, a very high knowledge in system design and OS scripting. minimum requirements: - assp: 64Bit OS, all SSD, 16GB RAM, 8 CPU cores, 64Bit Perl (multiple larger systems expected) - external high available enterprise database server - high available and very fast DNS-servers This ISP setup option is subject to become a payed licensed feature. Thomas DISCLAIMER: *** This email and any files transmitted with it may be confidential, legally privileged and protected in law and are intended solely for the use of the individual to whom it is addressed. This email was multiple times scanned for viruses. There should be no known virus in this email! *** -- Dive into the World of Parallel Programming The Go Parallel Website, sponsored by Intel and developed in partnership with Slashdot Media, is your hub for all things parallel software development, from weekly thought leadership blogs to news, videos, case studies, tutorials and more. Take a look and join the conversation now. http://goparallel.sourceforge.net/ ___ Assp-test mailing list Assp-test@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/assp-test
