Re: [foreman-dev] RFC idea: Templates rendering only through Proxies
>> Not true. Finish scripts are used by preseed (Debian/Ubuntu) for the >> PXE-based Yeah thanks. > +1 for calling something from post section (and pre and any other progress > indicator we can) to be able to show where we are during provisioning > stage... especially if we want to get smarter about error handing during > provisioning. Sure, we can extend our "built" call so it handles stages as well. But I do believe this does not require templates to be proxied at all. LZ -- You received this message because you are subscribed to the Google Groups "foreman-dev" group. To unsubscribe from this group and stop receiving emails from it, send an email to foreman-dev+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: [foreman-dev] RFC idea: Templates rendering only through Proxies
On Fri, Jan 27, 2017 at 12:23 AM, Greg Sutcliffe wrote: > I'll comment on the rest of this when I have time to think it over, > however I > do want to correct one point: > > On Thursday, 26 January 2017 10:27:47 CET Lukas Zapletal wrote: > > > This is only relevant to provision, iPXE and script templates, all the > > others are not affected by this. PXE templates already works this way, > > finish and user_data are injected via ssh/cloudinit. > > Not true. Finish scripts are used by preseed (Debian/Ubuntu) for the > PXE-based > provisioning process, and there were some discussions around moving the > kickstart to doing the same for parity (i.e. calling the finish template > from > %post). You'll need to therefore account for the differing use of finish > templates depending on provision method. > +1 for calling something from post section (and pre and any other progress indicator we can) to be able to show where we are during provisioning stage... especially if we want to get smarter about error handing during provisioning. Ohad > > Greg > -- > Greg Sutcliffe, Foreman Community Lead > IRC / Twitter: gwmngilfen > Diaspora: gwmngil...@joindiaspora.com > > -- > You received this message because you are subscribed to the Google Groups > "foreman-dev" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to foreman-dev+unsubscr...@googlegroups.com. > For more options, visit https://groups.google.com/d/optout. > -- You received this message because you are subscribed to the Google Groups "foreman-dev" group. To unsubscribe from this group and stop receiving emails from it, send an email to foreman-dev+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: [foreman-dev] RFC idea: Templates rendering only through Proxies
I'll comment on the rest of this when I have time to think it over, however I do want to correct one point: On Thursday, 26 January 2017 10:27:47 CET Lukas Zapletal wrote: > This is only relevant to provision, iPXE and script templates, all the > others are not affected by this. PXE templates already works this way, > finish and user_data are injected via ssh/cloudinit. Not true. Finish scripts are used by preseed (Debian/Ubuntu) for the PXE-based provisioning process, and there were some discussions around moving the kickstart to doing the same for parity (i.e. calling the finish template from %post). You'll need to therefore account for the differing use of finish templates depending on provision method. Greg -- Greg Sutcliffe, Foreman Community Lead IRC / Twitter: gwmngilfen Diaspora: gwmngil...@joindiaspora.com -- You received this message because you are subscribed to the Google Groups "foreman-dev" group. To unsubscribe from this group and stop receiving emails from it, send an email to foreman-dev+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout. signature.asc Description: This is a digitally signed message part.
Re: [foreman-dev] RFC idea: Templates rendering only through Proxies
I think this is a good idea; and it can be pushed into separating templates into a standalone service which would handle the whole template lifecycle, not just serving of pre-rendered templates (if eventually). Also agreed re: “built” ping, it’s a special call, although the responsibility for collecting this data can be moved to smart-proxy too. OT: HA smart-proxy setups are hard b/c proxy wasn’t engineered for HA, and different HA setups require different approaches (HA dhcp is different from HA dns, etc). It very much solvable, but it’s a fair bit of work and so far I was unable to get a clear answer about how important is to have such a feature. -d On Thu, Jan 26, 2017 at 3:21 PM, Lukas Zapletal wrote: > On Thu, Jan 26, 2017 at 11:16 AM, Timo Goebel wrote: >> How do you want to handle the "built"-Url? I think, you still need a >> callback to Foreman when a Host has finished provisioning. That basically >> voids most of the advantages (like improving reliability - btw: a Foreman ha >> is easy vs. a smart-proxy ha setup is hard). > > Yeah, sure, I noticed this too, but this built call is special - no > output, no need of determining template API endpoint etc. It's a very > simple case. > >> I do think, it's a good idea that Foreman knows the current state of an >> installation. We could even show valueable information on the provisioning >> progress or potential errors like with [2]. That would be a huge benefit for >> users, at least from my experience. > > My proposal does not change that, I believe. Template contents audit > is something I miss from the day one when I started with foreman and > pre-rendering could basically help with that - we could put whole > template content into audit record during build, so we can be sure > clients see the exactly same. > > -- > Later, > Lukas @lzap Zapletal > > -- > You received this message because you are subscribed to the Google Groups > "foreman-dev" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to foreman-dev+unsubscr...@googlegroups.com. > For more options, visit https://groups.google.com/d/optout. -- You received this message because you are subscribed to the Google Groups "foreman-dev" group. To unsubscribe from this group and stop receiving emails from it, send an email to foreman-dev+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: [foreman-dev] RFC idea: Templates rendering only through Proxies
On Thu, Jan 26, 2017 at 11:16 AM, Timo Goebel wrote: > How do you want to handle the "built"-Url? I think, you still need a > callback to Foreman when a Host has finished provisioning. That basically > voids most of the advantages (like improving reliability - btw: a Foreman ha > is easy vs. a smart-proxy ha setup is hard). Yeah, sure, I noticed this too, but this built call is special - no output, no need of determining template API endpoint etc. It's a very simple case. > I do think, it's a good idea that Foreman knows the current state of an > installation. We could even show valueable information on the provisioning > progress or potential errors like with [2]. That would be a huge benefit for > users, at least from my experience. My proposal does not change that, I believe. Template contents audit is something I miss from the day one when I started with foreman and pre-rendering could basically help with that - we could put whole template content into audit record during build, so we can be sure clients see the exactly same. -- Later, Lukas @lzap Zapletal -- You received this message because you are subscribed to the Google Groups "foreman-dev" group. To unsubscribe from this group and stop receiving emails from it, send an email to foreman-dev+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: [foreman-dev] RFC idea: Templates rendering only through Proxies
Hi, > our design of template proxying is not good I don't think, it's that bad. Your suggestion does have it's advantages, though. From a security perspective it would be great if the smart-proxy wouldn't need any calls to Foreman. > The downside is that Smart Proxy would be required in order to do templating If we pre-render all templates in Foreman and just deploy text files to the proxy, why do we need templating on the proxy? > There should not be any technical limitation, unless I miss something. How do you want to handle the "built"-Url? I think, you still need a callback to Foreman when a Host has finished provisioning. That basically voids most of the advantages (like improving reliability - btw: a Foreman ha is easy vs. a smart-proxy ha setup is hard). We currently setup puppetca orchestration when a Host requests the provisioning template. That would need to be changed first. Please see [1]. I do think, it's a good idea that Foreman knows the current state of an installation. We could even show valueable information on the provisioning progress or potential errors like with [2]. That would be a huge benefit for users, at least from my experience. Timo [1] https://github.com/theforeman/rfcs/pull/7/files [2] https://github.com/ShimShtein/foreman_build_history -- You received this message because you are subscribed to the Google Groups "foreman-dev" group. To unsubscribe from this group and stop receiving emails from it, send an email to foreman-dev+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[foreman-dev] RFC idea: Templates rendering only through Proxies
Hey, our design of template proxying is not good, I constantly need to solve problems setting it up. In order to have this working, both Templates and TFTP features needs to be turned on. This is confusing, then several Proxy settings need to be correctly set and since the names are also confusing, users fail to set them correctly: /etc/foreman-proxy/settings.d/templates.yml :template_url: http://ibm-x3690x5-01.xxx.com:8000 /etc/foreman-proxy/settings.yml :foreman_url: https//ibm-x3690x5-01.xxx.com :https_port: 9090 :http_port: 8000 Here is the flow of a kickstart request: During template rendering (PXELinux that contains KS URL): Foreman -> Proxy (HTTP/8000) - reads template_url setting and that is rendered into the template for the OS installer During provisioning (installer tries to access the URL given in PXELinux template): Installer -> Proxy (HTTP/8000) -> Foreman (HTTPS/443) - reads content of kickstart template I was thinking if it's the time to simply stop doing on-the-fly template rendering in Foreman and start copying them onto all relevant Proxies during Build phase. The downside is that Smart Proxy would be required in order to do templating, on the other hand this simplifies setup and it also errors out early if there was a rendering issue (we had to implement a workaround - we render them into /dev/null during Build phase). Smart Proxy could store all the associated templates in a directory as plain files and hand them over to clients. Therefore no database needed. Token and expiration handling can be done in Proxy as well (file has a creation time). This also improves reliability (when Forman server is down, clients could still reboot). This is only relevant to provision, iPXE and script templates, all the others are not affected by this. PXE templates already works this way, finish and user_data are injected via ssh/cloudinit. There should not be any technical limitation, unless I miss something. Opinions? -- Later, Lukas @lzap Zapletal -- You received this message because you are subscribed to the Google Groups "foreman-dev" group. To unsubscribe from this group and stop receiving emails from it, send an email to foreman-dev+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
