Re: [Bioc-devel] Build error due to TensorFlow installation
how about TF for platforms that support it and C++ for those that don't, eventually migrating away from C++ as support grows? a little messy for the moment but let's get some mileage on the TF solution ASAP On Thu, Sep 5, 2019 at 9:21 AM Dirmeier Simon wrote: > Hi Herve, > > > All this means that if you replace some old C++ code with TensorFlow > > then we will need to mark your package as unavailable on Windows and > > Mac, at least for now. Sorry. I wonder if there was something wrong > > with this old C++ code. I would recommend that you stick to it if you > can. > > > The code is fine, but still an impractical complexity that doesn't need > to exist: > > 1) It's hard to extend and read for others. > > 2) It needs a custom configure.ac. > > 3) Extending the package to other models/families is a huge pain, as one > needs to derive custom coordinate descents (or other optimizers for that > matter) for each. > > On the other side: > > 1) TF allowed me to replace like 5000 lines of source code with 100 > lines of R. > > 2) TF allows to easily extend with other models with just a few lines. > > 3) I don't need a huge test suite. > > 4) On GPUs it's a huge speedup. > > So, for now I'll revert the changes back on Bioc devel and continue > development on another branch. > > Cheers, > > S > > > Am 04.09.19 um 16:53 schrieb Pages, Herve: > > > > Hi Simon, > > > > On 9/3/19 09:11, Simon Dirmeier wrote: > >> ... > >> Do you think it would be possible to install TensorFlow and > >> TensorFlow-Probability on the builders? I'd assume that many would > >> profit from that. > >> > > As Lori mentioned at the end of her email (see below), we can't make > > the tensorflow Python module available on our Windows builders at the > > moment because we need to update Python to Python 3 on these machines > > first (AFAIK tensorflow is only available for Python 3 on Windows). > > This is something that we are currently working on. > > > > As for the Mac builders, we have tensorflow there but unfortunately > > it's an old version because recent versions of are broken on El > > Capitan (this is the Mac OS version that, for various reasons, we are > > stuck with at the moment). This prevents us from installing the > > tensorflow_probability module which requires a recent version of > > tensorflow. > > > > The tensorflow and tensorflow_probability modules are available on our > > Linux builders. > > > > All this means that if you replace some old C++ code with TensorFlow > > then we will need to mark your package as unavailable on Windows and > > Mac, at least for now. Sorry. I wonder if there was something wrong > > with this old C++ code. I would recommend that you stick to it if you > can. > > > > Best, > > > > H. > > > > > >>> Right now tensorflow is unavailable on our windows server (tokay1)�as > >>> we have not updated to python 3 (but will be in the near future)� The > >>> windows error can be ignored for now. > >>> > >>> > >>> > >>> Lori Shepherd > >>> > >>> Bioconductor Core Team > >>> > >>> Roswell Park Cancer Institute > >>> > >>> Department of Biostatistics & Bioinformatics > >>> > >>> Elm & Carlton Streets > >>> > >>> Buffalo, New York 14263 > >>> > >>> > > >>> *From:* Bioc-devel on behalf of > >>> Simon Dirmeier > >>> *Sent:* Monday, September 2, 2019 6:08:45 AM > >>> *To:*bioc-devel@r-project.org > >>> *Subject:* [Bioc-devel] Build error due to TensorFlow installation > >>> Dear all, > >>> > >>> since I replaced some old C++ code with TensorFlow I am getting some > >>> build errors on merida1 and tokay1 regarding installation (even though > I > >>> install TF and TF Probability during /.onLoad/) > >>> > >>> > https://urldefense.proofpoint.com/v2/url?u=http-3A__bioconductor.org_checkResults_devel_bioc-2DLATEST_netReg_merida1-2Dbuildsrc.html&d=DwIFaQ&c=eRAMFD45gAfqt84VtBcfhQ&r=BK7q3XeAvimeWdGbWY_wJYbW0WYiZvSXAJJKaaPhzWA&m=Iy8WHNH6twF-8QkJjKa7MOIXm9mYjGg4pBlLb2CGiq8&s=wjC8b2uCm0PJgQ1LRO_CTrqNvfGKpEKN0N6QN23Sbd0&e= > > >>> > >>> Does anyone know how I can fix this or did anyone use TF with > >>&
Re: [Bioc-devel] Build error due to TensorFlow installation
Hi Herve, > All this means that if you replace some old C++ code with TensorFlow > then we will need to mark your package as unavailable on Windows and > Mac, at least for now. Sorry. I wonder if there was something wrong > with this old C++ code. I would recommend that you stick to it if you can. > The code is fine, but still an impractical complexity that doesn't need to exist: 1) It's hard to extend and read for others. 2) It needs a custom configure.ac. 3) Extending the package to other models/families is a huge pain, as one needs to derive custom coordinate descents (or other optimizers for that matter) for each. On the other side: 1) TF allowed me to replace like 5000 lines of source code with 100 lines of R. 2) TF allows to easily extend with other models with just a few lines. 3) I don't need a huge test suite. 4) On GPUs it's a huge speedup. So, for now I'll revert the changes back on Bioc devel and continue development on another branch. Cheers, S Am 04.09.19 um 16:53 schrieb Pages, Herve: > > Hi Simon, > > On 9/3/19 09:11, Simon Dirmeier wrote: >> ... >> Do you think it would be possible to install TensorFlow and >> TensorFlow-Probability on the builders? I'd assume that many would >> profit from that. >> > As Lori mentioned at the end of her email (see below), we can't make > the tensorflow Python module available on our Windows builders at the > moment because we need to update Python to Python 3 on these machines > first (AFAIK tensorflow is only available for Python 3 on Windows). > This is something that we are currently working on. > > As for the Mac builders, we have tensorflow there but unfortunately > it's an old version because recent versions of are broken on El > Capitan (this is the Mac OS version that, for various reasons, we are > stuck with at the moment). This prevents us from installing the > tensorflow_probability module which requires a recent version of > tensorflow. > > The tensorflow and tensorflow_probability modules are available on our > Linux builders. > > All this means that if you replace some old C++ code with TensorFlow > then we will need to mark your package as unavailable on Windows and > Mac, at least for now. Sorry. I wonder if there was something wrong > with this old C++ code. I would recommend that you stick to it if you can. > > Best, > > H. > > >>> Right now tensorflow is unavailable on our windows server (tokay1)�as >>> we have not updated to python 3 (but will be in the near future)� The >>> windows error can be ignored for now. >>> >>> >>> >>> Lori Shepherd >>> >>> Bioconductor Core Team >>> >>> Roswell Park Cancer Institute >>> >>> Department of Biostatistics & Bioinformatics >>> >>> Elm & Carlton Streets >>> >>> Buffalo, New York 14263 >>> >>> >>> *From:* Bioc-devel on behalf of >>> Simon Dirmeier >>> *Sent:* Monday, September 2, 2019 6:08:45 AM >>> *To:*bioc-devel@r-project.org >>> *Subject:* [Bioc-devel] Build error due to TensorFlow installation >>> Dear all, >>> >>> since I replaced some old C++ code with TensorFlow I am getting some >>> build errors on merida1 and tokay1 regarding installation (even though I >>> install TF and TF Probability during /.onLoad/) >>> >>> https://urldefense.proofpoint.com/v2/url?u=http-3A__bioconductor.org_checkResults_devel_bioc-2DLATEST_netReg_merida1-2Dbuildsrc.html&d=DwIFaQ&c=eRAMFD45gAfqt84VtBcfhQ&r=BK7q3XeAvimeWdGbWY_wJYbW0WYiZvSXAJJKaaPhzWA&m=Iy8WHNH6twF-8QkJjKa7MOIXm9mYjGg4pBlLb2CGiq8&s=wjC8b2uCm0PJgQ1LRO_CTrqNvfGKpEKN0N6QN23Sbd0&e= >>> >>> >>> Does anyone know how I can fix this or did anyone use TF with >>> Bioconductor so far? >>> >>> Many thanks in advance. >>> >>> Best, >>> >>> Simon >>> >>> >>> >>> >>> >>> ��� [[alternative HTML version deleted]] >>> >>> ___ >>> Bioc-devel@r-project.org mailing list >>> https://urldefense.proofpoint.com/v2/url?u=https-3A__stat.ethz.ch_mailman_listinfo_bioc-2Ddevel&d=DwIFaQ&c=eRAMFD45gAfqt84VtBcfhQ&r=BK7q3XeAvimeWdGbWY_wJYbW0WYiZvSXAJJKaaPhzWA&m=Iy8WHNH6twF-8QkJjKa7MOIXm9mYjGg4pBlLb2CGiq8&s=98vYdk5e2ryxlWN99O68ty2dUHnoE23VPU9TwSF8rPc&e= >>> >>> >>> This email message may contain legally privileged
Re: [Bioc-devel] Build error due to TensorFlow installation
Hi Simon, On 9/3/19 09:11, Simon Dirmeier wrote: ... Do you think it would be possible to install TensorFlow and TensorFlow-Probability on the builders? I'd assume that many would profit from that. As Lori mentioned at the end of her email (see below), we can't make the tensorflow Python module available on our Windows builders at the moment because we need to update Python to Python 3 on these machines first (AFAIK tensorflow is only available for Python 3 on Windows). This is something that we are currently working on. As for the Mac builders, we have tensorflow there but unfortunately it's an old version because recent versions of are broken on El Capitan (this is the Mac OS version that, for various reasons, we are stuck with at the moment). This prevents us from installing the tensorflow_probability module which requires a recent version of tensorflow. The tensorflow and tensorflow_probability modules are available on our Linux builders. All this means that if you replace some old C++ code with TensorFlow then we will need to mark your package as unavailable on Windows and Mac, at least for now. Sorry. I wonder if there was something wrong with this old C++ code. I would recommend that you stick to it if you can. Best, H. Right now tensorflow is unavailable on our windows server (tokay1)�as we have not updated to python 3 (but will be in the near future)� The windows error can be ignored for now. Lori Shepherd Bioconductor Core Team Roswell Park Cancer Institute Department of Biostatistics & Bioinformatics Elm & Carlton Streets Buffalo, New York 14263 *From:* Bioc-devel <mailto:bioc-devel-boun...@r-project.org> on behalf of Simon Dirmeier <mailto:simon.dirme...@web.de> *Sent:* Monday, September 2, 2019 6:08:45 AM *To:* bioc-devel@r-project.org<mailto:bioc-devel@r-project.org> <mailto:bioc-devel@r-project.org> *Subject:* [Bioc-devel] Build error due to TensorFlow installation Dear all, since I replaced some old C++ code with TensorFlow I am getting some build errors on merida1 and tokay1 regarding installation (even though I install TF and TF Probability during /.onLoad/) https://urldefense.proofpoint.com/v2/url?u=http-3A__bioconductor.org_checkResults_devel_bioc-2DLATEST_netReg_merida1-2Dbuildsrc.html&d=DwIFaQ&c=eRAMFD45gAfqt84VtBcfhQ&r=BK7q3XeAvimeWdGbWY_wJYbW0WYiZvSXAJJKaaPhzWA&m=Iy8WHNH6twF-8QkJjKa7MOIXm9mYjGg4pBlLb2CGiq8&s=wjC8b2uCm0PJgQ1LRO_CTrqNvfGKpEKN0N6QN23Sbd0&e= Does anyone know how I can fix this or did anyone use TF with Bioconductor so far? Many thanks in advance. Best, Simon ��� [[alternative HTML version deleted]] ___ Bioc-devel@r-project.org<mailto:Bioc-devel@r-project.org> mailing list https://urldefense.proofpoint.com/v2/url?u=https-3A__stat.ethz.ch_mailman_listinfo_bioc-2Ddevel&d=DwIFaQ&c=eRAMFD45gAfqt84VtBcfhQ&r=BK7q3XeAvimeWdGbWY_wJYbW0WYiZvSXAJJKaaPhzWA&m=Iy8WHNH6twF-8QkJjKa7MOIXm9mYjGg4pBlLb2CGiq8&s=98vYdk5e2ryxlWN99O68ty2dUHnoE23VPU9TwSF8rPc&e= This email message may contain legally privileged and/or confidential information. If you are not the intended recipient(s), or the employee or agent responsible for the delivery of this message to the intended recipient(s), you are hereby notified that any disclosure, copying, distribution, or use of this email message is prohibited. If you have received this message in error, please notify the sender immediately by e-mail and delete this email message from your computer. Thank you. [[alternative HTML version deleted]] ___ Bioc-devel@r-project.org<mailto:Bioc-devel@r-project.org> mailing list https://urldefense.proofpoint.com/v2/url?u=https-3A__stat.ethz.ch_mailman_listinfo_bioc-2Ddevel&d=DwICAg&c=eRAMFD45gAfqt84VtBcfhQ&r=BK7q3XeAvimeWdGbWY_wJYbW0WYiZvSXAJJKaaPhzWA&m=Iy8WHNH6twF-8QkJjKa7MOIXm9mYjGg4pBlLb2CGiq8&s=98vYdk5e2ryxlWN99O68ty2dUHnoE23VPU9TwSF8rPc&e= -- Hervé Pagès Program in Computational Biology Division of Public Health Sciences Fred Hutchinson Cancer Research Center 1100 Fairview Ave. N, M1-B514 P.O. Box 19024 Seattle, WA 98109-1024 E-mail: hpa...@fredhutch.org<mailto:hpa...@fredhutch.org> Phone: (206) 667-5791 Fax:(206) 667-1319 [[alternative HTML version deleted]] ___ Bioc-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/bioc-devel
Re: [Bioc-devel] Build error due to TensorFlow installation
Hi Lori, thanks for the reply. > In general we never recommended automatically installing anything on a > users system without them knowing (ie.� You shouldn't be installing > modules and packages in the onLoad and this should be removed) It > should be included in the README and as an�INSTALL file included > in�the package top level directory. > I see, thanks! I agree it's kinda dodgy but I didn't know a better way to pass the BioC checks. I changed it accordingly. > Any needed module or system dependency should be assumed to be > installed correctly on a users system�and if you need the builders to > have something installed a request made via this mailing list (Please > let us know if you think something is missing and needs to be > installed on the daily builders�- we would generally then look for the > INSTALL file on the git repository for installation instructions if it > was justifiable) > Do you think it would be possible to install TensorFlow and TensorFlow-Probability on the builders? I'd assume that many would profit from that. Best regards, Simon > Right now tensorflow is unavailable on our windows server (tokay1)�as > we have not updated to python 3 (but will be in the near future)� The > windows error can be ignored for now. > > > > Lori Shepherd > > Bioconductor Core Team > > Roswell Park Cancer Institute > > Department of Biostatistics & Bioinformatics > > Elm & Carlton Streets > > Buffalo, New York 14263 > > > *From:* Bioc-devel on behalf of > Simon Dirmeier > *Sent:* Monday, September 2, 2019 6:08:45 AM > *To:* bioc-devel@r-project.org > *Subject:* [Bioc-devel] Build error due to TensorFlow installation > Dear all, > > since I replaced some old C++ code with TensorFlow I am getting some > build errors on merida1 and tokay1 regarding installation (even though I > install TF and TF Probability during /.onLoad/) > > http://bioconductor.org/checkResults/devel/bioc-LATEST/netReg/merida1-buildsrc.html > > Does anyone know how I can fix this or did anyone use TF with > Bioconductor so far? > > Many thanks in advance. > > Best, > > Simon > > > > > > ��� [[alternative HTML version deleted]] > > ___ > Bioc-devel@r-project.org mailing list > https://stat.ethz.ch/mailman/listinfo/bioc-devel > > This email message may contain legally privileged and/or confidential > information. If you are not the intended recipient(s), or the employee > or agent responsible for the delivery of this message to the intended > recipient(s), you are hereby notified that any disclosure, copying, > distribution, or use of this email message is prohibited. If you have > received this message in error, please notify the sender immediately > by e-mail and delete this email message from your computer. Thank you. [[alternative HTML version deleted]] ___ Bioc-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/bioc-devel
Re: [Bioc-devel] Build error due to TensorFlow installation
Perhaps others will have more to say or disagree but... In general we never recommended automatically installing anything on a users system without them knowing (ie. You shouldn't be installing modules and packages in the onLoad and this should be removed) It should be included in the README and as an INSTALL file included in the package top level directory. Any needed module or system dependency should be assumed to be installed correctly on a users system and if you need the builders to have something installed a request made via this mailing list (Please let us know if you think something is missing and needs to be installed on the daily builders - we would generally then look for the INSTALL file on the git repository for installation instructions if it was justifiable) Right now tensorflow is unavailable on our windows server (tokay1) as we have not updated to python 3 (but will be in the near future) The windows error can be ignored for now. Lori Shepherd Bioconductor Core Team Roswell Park Cancer Institute Department of Biostatistics & Bioinformatics Elm & Carlton Streets Buffalo, New York 14263 From: Bioc-devel on behalf of Simon Dirmeier Sent: Monday, September 2, 2019 6:08:45 AM To: bioc-devel@r-project.org Subject: [Bioc-devel] Build error due to TensorFlow installation Dear all, since I replaced some old C++ code with TensorFlow I am getting some build errors on merida1 and tokay1 regarding installation (even though I install TF and TF Probability during /.onLoad/) http://bioconductor.org/checkResults/devel/bioc-LATEST/netReg/merida1-buildsrc.html Does anyone know how I can fix this or did anyone use TF with Bioconductor so far? Many thanks in advance. Best, Simon [[alternative HTML version deleted]] ___ Bioc-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/bioc-devel This email message may contain legally privileged and/or confidential information. If you are not the intended recipient(s), or the employee or agent responsible for the delivery of this message to the intended recipient(s), you are hereby notified that any disclosure, copying, distribution, or use of this email message is prohibited. If you have received this message in error, please notify the sender immediately by e-mail and delete this email message from your computer. Thank you. [[alternative HTML version deleted]] ___ Bioc-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/bioc-devel
[Bioc-devel] Build error due to TensorFlow installation
Dear all, since I replaced some old C++ code with TensorFlow I am getting some build errors on merida1 and tokay1 regarding installation (even though I install TF and TF Probability during /.onLoad/) http://bioconductor.org/checkResults/devel/bioc-LATEST/netReg/merida1-buildsrc.html Does anyone know how I can fix this or did anyone use TF with Bioconductor so far? Many thanks in advance. Best, Simon [[alternative HTML version deleted]] ___ Bioc-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/bioc-devel
