Re: [pve-devel] [PATCH conntrack-tool v2 2/5] add packaging support
On 03.02.21 15:25, Mira Limbeck wrote:
> Signed-off-by: Mira Limbeck
> ---
> v2:
> - unchanged
Some nits/comments inline.
You probably adapted this from proxmox-backup so my comments may hold true
there too.
>
> .cargo/config| 5
> Makefile | 63
> debian/changelog | 5
> debian/copyright | 16 +++
> debian/debcargo.toml | 17
> debian/docs | 1 +
> 6 files changed, 107 insertions(+)
> create mode 100644 .cargo/config
> create mode 100644 Makefile
> create mode 100644 debian/changelog
> create mode 100644 debian/copyright
> create mode 100644 debian/debcargo.toml
> create mode 100644 debian/docs
>
> diff --git a/.cargo/config b/.cargo/config
> new file mode 100644
> index 000..3b5b6e4
> --- /dev/null
> +++ b/.cargo/config
> @@ -0,0 +1,5 @@
> +[source]
> +[source.debian-packages]
> +directory = "/usr/share/cargo/registry"
> +[source.crates-io]
> +replace-with = "debian-packages"
> diff --git a/Makefile b/Makefile
> new file mode 100644
> index 000..8ea2f8a
> --- /dev/null
> +++ b/Makefile
> @@ -0,0 +1,63 @@
> +include /usr/share/dpkg/pkg-info.mk
> +include /usr/share/dpkg/architecture.mk
> +
> +PACKAGE=pve-conntrack-tool
> +
> +GITVERSION:=$(shell git rev-parse HEAD)
> +
> +DEB=${PACKAGE}_${DEB_VERSION_UPSTREAM_REVISION}_${DEB_BUILD_ARCH}.deb
> +DSC=rust-${PACKAGE}_${DEB_VERSION_UPSTREAM_REVISION}.dsc
> +
> +ifeq ($(BUILD_MODE), release)
> +CARGO_BUILD_ARGS += --release
> +COMPILEDIR := target/release
> +else
> +COMPILEDIR := target/debug
> +endif
> +
> +all: cargo-build $(SUBDIRS)
> +
> +.PHONY: cargo-build
> +cargo-build:
> + cargo build $(CARGO_BUILD_ARGS)
> +
> +.PHONY: build
> +build:
We normally try to use a more Debian conform build directory, like:
BUILDDIR ?= ${PACKAGE}-${DEB_VERSION_UPSTREAM}
This way one can also easily set their own via a make parameter.
> + rm -rf build
> + debcargo package \
> + --config debian/debcargo.toml \
> + --changelog-ready \
> + --no-overlay-write-back \
> + --directory build \
> + $(PACKAGE) \
> + $(shell dpkg-parsechangelog -l debian/changelog -SVersion | sed -e
> 's/-.*//')
> + rm build/Cargo.lock
> + find build/debian -name "*.hint" -delete
can we do all in a temporary directory and do a rename at the end? making this
target actin a bit more "atomic".
> +
> +.PHONY: deb
> +deb: $(DEB)
> +$(DEB): build
> + cd build; dpkg-buildpackage -b -us -uc --no-pre-clean
> --build-profiles=nodoc
is the `nodoc` actually required? Is it always generated by debcargo, as I do
not
see any d/control like reference to it.
> + lintian $(DEB)
> +
> +.PHONY: dsc
> +dsc: $(DSC)
> +$(DSC): build
> + cd build; dpkg-buildpackage -S -us -uc -d -nc
> + lintian $(DSC)
> +
> +.PHONY: dinstall
> +dinstall: ${DEB}
> + dpkg -i ${DEB}
> +
> +.PHONY: upload
> +upload: ${DEB} ${DBG_DEB}
> + tar cf - ${DEB} ${DBG_DEB}| ssh -X repo...@repo.proxmox.com -- upload
> --product pve --dist buster --arch ${DEB_BUILD_ARCH}
> +
> +.PHONY: distclean
> +distclean: clean
> +
> +.PHONY: clean
> +clean:
> + rm -rf *.deb ${PACKAGE}-* *.buildinfo *.changes *.dsc
> rust-${PACKAGE}_*.tar.?z build/
> + find . -name '*~' -exec rm {} ';'
> diff --git a/debian/changelog b/debian/changelog
> new file mode 100644
> index 000..700e739
> --- /dev/null
> +++ b/debian/changelog
> @@ -0,0 +1,5 @@
> +rust-pve-conntrack-tool (1.0.0-1) pve; urgency=medium
> +
> + * Initial release.
> +
> + -- Mira Limbeck Thu, 08 Oct 2020 14:04:19 +0200
> diff --git a/debian/copyright b/debian/copyright
> new file mode 100644
> index 000..4d6f36a
> --- /dev/null
> +++ b/debian/copyright
> @@ -0,0 +1,16 @@
> +Copyright (C) 2020 Proxmox Server Solutions GmbH
2020 - 2021
> +
> +This software is written by Proxmox Server Solutions GmbH
>
> +
> +This program is free software: you can redistribute it and/or modify
> +it under the terms of the GNU Affero General Public License as published by
> +the Free Software Foundation, either version 3 of the License, or
> +(at your option) any later version.
> +
> +This program is distributed in the hope that it will be useful,
> +but WITHOUT ANY WARRANTY; without even the implied warranty of
> +MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
> +GNU Affero General Public License for more details.
> +
> +You should have received a copy of the GNU Affero General Public License
> +along with this program. If not, see
[pve-devel] [PATCH conntrack-tool v2 1/5] initial commit
Dumping conntrack information and importing conntrack information works
for IPv4 and IPv6. No filtering is supported for now. pve-conntrack-tool
will always return both IPv4 and IPv6 conntracks together.
Conntracks are serialized as JSON and printed on STDOUT line by line
with one line containing one conntrack. When inserting data is read
from STDIN line by line and expected to be one JSON object per line
representing the conntrack.
Currently some conntrack attributes are not supported. These are
HELPER_INFO, CONNLABELS and CONNLABELS_MASK. The reason for this is that
handling of variable length attributes does not seem to be correctly
implemented in libnetfilter_conntrack. To fix this we would probably have
to use libmnl directly.
Conntracks containing protonum 2 (IGMP) are ignored in the dump as
they can't be inserted using libnetfilter_conntrack (conntrack-tools'
conntrack also exhibits the same behavior).
Expectation support, which is necessary for FTP and other protocols, is
not yet implemented.
Signed-off-by: Mira Limbeck
---
v2:
- changed Conntracks to Socket
- reworked a lot of the code for less code duplication
- reduced usage of 'unsafe'
- added/changed things based on @Wobu's suggestions (off-list)
Cargo.toml | 14 ++
src/main.rs| 488 +
src/mnl.rs | 132 ++
src/netfilter_conntrack.rs | 168 +
4 files changed, 802 insertions(+)
create mode 100644 Cargo.toml
create mode 100644 src/main.rs
create mode 100644 src/mnl.rs
create mode 100644 src/netfilter_conntrack.rs
diff --git a/Cargo.toml b/Cargo.toml
new file mode 100644
index 000..3e3851a
--- /dev/null
+++ b/Cargo.toml
@@ -0,0 +1,14 @@
+[package]
+name = "pve-conntrack-tool"
+version = "1.0.0"
+authors = ["Mira Limbeck "]
+edition = "2018"
+license = "AGPL-3"
+
+exclude = [ "build", "debian" ]
+
+[dependencies]
+anyhow = "1.0.26"
+libc = "0.2.69"
+serde = { version = "1.0.106", features = ["derive"] }
+serde_json = "1.0.41"
diff --git a/src/main.rs b/src/main.rs
new file mode 100644
index 000..2137556
--- /dev/null
+++ b/src/main.rs
@@ -0,0 +1,488 @@
+mod mnl;
+mod netfilter_conntrack;
+
+use std::convert::TryInto;
+use std::ffi::CString;
+use std::io::{stdin, BufRead, BufReader};
+use std::os::unix::ffi::OsStringExt;
+use std::ptr::NonNull;
+
+use anyhow::{bail, format_err, Result};
+use mnl::*;
+use netfilter_conntrack::*;
+use serde::{Deserialize, Serialize};
+
+fn main() -> Result<()> {
+let args = std::env::args_os()
+.map(|os| String::from_utf8(os.into_vec()))
+.try_fold(Vec::new(), |mut args, s| match s {
+Ok(s) => {
+args.push(s);
+Ok(args)
+}
+Err(err) => bail!("Invalid UTF8 argument: {}", err),
+})?;
+if args.len() != 2 {
+bail!("Either 'dump' or 'insert' command required.");
+}
+
+let mut socket = Socket::open()?;
+
+if args[1] == "dump" {
+let mut cts = Vec::new();
+socket
+.query_conntracks(&mut cts)
+.map_err(|err| format_err!("Error querying conntracks: {}", err))?;
+
+for ct in cts.iter() {
+match serde_json::to_string(ct) {
+Ok(s) => println!("{}", s),
+Err(err) => {
+eprintln!("Failed to serialize conntrack: {}", err);
+break;
+}
+}
+}
+} else if args[1] == "insert" {
+for line in BufReader::new(stdin())
+.lines()
+.map(|line| line.unwrap_or_else(|_| "".to_string()))
+{
+let ct: Conntrack = match serde_json::from_str(&line) {
+Ok(ct) => ct,
+Err(err) => {
+eprintln!("Failed to deserialize conntrack: {}", err);
+break;
+}
+};
+if let Err(err) = socket.insert_conntrack(ct) {
+eprintln!("Error inserting conntrack: {}", err);
+}
+}
+} else {
+bail!("Unknown command: {}", args[1]);
+}
+
+Ok(())
+}
+
+extern "C" fn query_cts_cb(nlh: *const libc::nlmsghdr, data_ptr: *mut
libc::c_void) -> libc::c_int {
+let ct = unsafe { nfct_new() };
+unsafe {
+nfct_nlmsg_parse(nlh, ct);
+}
+
+if let Some(conntrack) = parse_conntrack(ct) {
+let cts: &mut Vec = unsafe { &mut *(data_ptr as *mut
Vec) };
+cts.push(conntrack);
+}
+
+unsafe {
+nfct_destroy(ct);
+}
+
+MNL_CB_OK
+}
+
+const CONNTRACK_QUERY_MSG_TYPE: u16 =
+((libc::NFNL_SUBSYS_CTNETLINK << 8) | IPCTNL_MSG_CT_GET) as u16;
+const CONNTRACK_QUERY_FLAGS: u16 =
+(libc::NLM_F_ACK | libc::NLM_F_REQUEST | libc::NLM_F_DUMP) as u16;
+const CONNTRACK_INSERT_MSG_TYPE: u16 =
+((libc::NFNL_SUBSYS_CTNETLINK << 8) | IPCTNL_MSG_CT_NEW) as u16;
+const CONNTRACK_INSERT_FLAGS: u16 =
+(libc::NLM_F_ACK | l
[pve-devel] [PATCH conntrack-tool v2 3/5] add expectation support
Expectation support requires net.netfilter.nf_conntrack_helper to be set
to 1. In addition the helper modules have to be loaded as well. In the
tests nf_conntrack_ftp was used as helper.
Together with expectation support, string attribute support is also
added. Some functions which are conntrack specific are renamed to
contain 'conntrack' in their names.
Signed-off-by: Mira Limbeck
---
v2:
- mostly the same changes as for patch 1
src/main.rs| 249 +++--
src/netfilter_conntrack.rs | 44 +++
2 files changed, 285 insertions(+), 8 deletions(-)
diff --git a/src/main.rs b/src/main.rs
index 2137556..79779ff 100644
--- a/src/main.rs
+++ b/src/main.rs
@@ -43,20 +43,37 @@ fn main() -> Result<()> {
}
}
}
+
+let mut exps = Vec::new();
+socket
+.query_expects(&mut exps)
+.map_err(|err| format_err!("Error querying expects: {}", err))?;
+
+for exp in exps.iter() {
+match serde_json::to_string(exp) {
+Ok(s) => println!("{}", s),
+Err(err) => {
+eprintln!("Failed to serialize expect: {}", err);
+break;
+}
+}
+}
} else if args[1] == "insert" {
for line in BufReader::new(stdin())
.lines()
.map(|line| line.unwrap_or_else(|_| "".to_string()))
{
-let ct: Conntrack = match serde_json::from_str(&line) {
-Ok(ct) => ct,
-Err(err) => {
-eprintln!("Failed to deserialize conntrack: {}", err);
-break;
+if let Ok(ct) = serde_json::from_str::(&line) {
+if let Err(err) = socket.insert_conntrack(ct) {
+eprintln!("Error inserting conntrack: {}", err);
}
-};
-if let Err(err) = socket.insert_conntrack(ct) {
-eprintln!("Error inserting conntrack: {}", err);
+} else if let Ok(exp) = serde_json::from_str::(&line) {
+if let Err(err) = socket.insert_expect(exp) {
+eprintln!("Error inserting expect: {}", err);
+}
+} else {
+eprintln!("Failed to deserialize input: {}", line);
+break;
}
}
} else {
@@ -92,6 +109,13 @@ const CONNTRACK_INSERT_MSG_TYPE: u16 =
((libc::NFNL_SUBSYS_CTNETLINK << 8) | IPCTNL_MSG_CT_NEW) as u16;
const CONNTRACK_INSERT_FLAGS: u16 =
(libc::NLM_F_ACK | libc::NLM_F_REQUEST | libc::NLM_F_CREATE) as u16;
+const EXPECT_QUERY_MSG_TYPE: u16 =
+((libc::NFNL_SUBSYS_CTNETLINK_EXP << 8) | IPCTNL_MSG_EXP_GET) as u16;
+const EXPECT_QUERY_FLAGS: u16 = (libc::NLM_F_ACK | libc::NLM_F_REQUEST |
libc::NLM_F_DUMP) as u16;
+const EXPECT_INSERT_MSG_TYPE: u16 =
+((libc::NFNL_SUBSYS_CTNETLINK_EXP << 8) | IPCTNL_MSG_EXP_NEW) as u16;
+const EXPECT_INSERT_FLAGS: u16 =
+(libc::NLM_F_ACK | libc::NLM_F_REQUEST | libc::NLM_F_CREATE) as u16;
pub struct Socket {
socket: NonNull,
@@ -176,6 +200,88 @@ impl Socket {
Ok(())
}
+fn query_expects(&mut self, exps: &mut Vec) -> Result<()> {
+let seq = self.seq();
+self.query_expects_impl(exps, seq, libc::AF_INET as _)?;
+let seq = self.seq();
+self.query_expects_impl(exps, seq, libc::AF_INET6 as _)?;
+Ok(())
+}
+
+fn query_expects_impl(&mut self, exps: &mut Vec, seq: u32, proto:
u8) -> Result<()> {
+let mut buf = [0u8; MNL_SOCKET_DUMP_SIZE as _];
+let hdr = build_msg_header(
+buf.as_mut_ptr() as _,
+EXPECT_QUERY_MSG_TYPE,
+EXPECT_QUERY_FLAGS,
+seq,
+proto,
+);
+self.send_and_receive(hdr, 0, Some(query_exp_cb), exps as *mut
Vec as _)
+}
+
+fn insert_expect(&mut self, exp: Expect) -> Result<()> {
+let proto = if exp.is_ipv6() {
+libc::AF_INET6 as u8
+} else {
+libc::AF_INET as u8
+};
+
+let mut buf = [0u8; MNL_SOCKET_BUFFER_SIZE as _];
+let hdr = build_msg_header(
+buf.as_mut_ptr() as _,
+EXPECT_INSERT_MSG_TYPE,
+EXPECT_INSERT_FLAGS,
+self.seq(),
+proto,
+);
+
+let exph = unsafe { nfexp_new() };
+if exph.is_null() {
+bail!("Failed to create new expect object");
+}
+
+let mut strings = Vec::new();
+let mut cts = Vec::new();
+for attr in exp.attributes {
+match attr.value {
+ExpectAttrValue::CT(ct) => unsafe {
+let (ct, mut s) = build_conntrack(ct)?;
+nfexp_set_attr(exph, attr.key, ct as _);
+strings.append(&mut s);
+cts.push(ct);
+},
+ExpectAttrValue::U8(v) => unsafe {
+
[pve-devel] [PATCH conntrack-tool v2 5/5] replace C callback with closures
Internally we still have to use a C callback, but all it does is forward
to the closure we pass to it.
Signed-off-by: Mira Limbeck
---
v2:
- new addition
src/main.rs | 208 +++-
1 file changed, 109 insertions(+), 99 deletions(-)
diff --git a/src/main.rs b/src/main.rs
index 79779ff..0930f92 100644
--- a/src/main.rs
+++ b/src/main.rs
@@ -83,24 +83,6 @@ fn main() -> Result<()> {
Ok(())
}
-extern "C" fn query_cts_cb(nlh: *const libc::nlmsghdr, data_ptr: *mut
libc::c_void) -> libc::c_int {
-let ct = unsafe { nfct_new() };
-unsafe {
-nfct_nlmsg_parse(nlh, ct);
-}
-
-if let Some(conntrack) = parse_conntrack(ct) {
-let cts: &mut Vec = unsafe { &mut *(data_ptr as *mut
Vec) };
-cts.push(conntrack);
-}
-
-unsafe {
-nfct_destroy(ct);
-}
-
-MNL_CB_OK
-}
-
const CONNTRACK_QUERY_MSG_TYPE: u16 =
((libc::NFNL_SUBSYS_CTNETLINK << 8) | IPCTNL_MSG_CT_GET) as u16;
const CONNTRACK_QUERY_FLAGS: u16 =
@@ -169,7 +151,20 @@ impl Socket {
seq,
proto,
);
-self.send_and_receive(hdr, 0, Some(query_cts_cb), cts as *mut
Vec as _)
+self.send_and_receive(hdr, 0, |nlh| {
+let ct = unsafe { nfct_new() };
+unsafe {
+nfct_nlmsg_parse(nlh, ct);
+}
+
+if let Some(conntrack) = parse_conntrack(ct) {
+cts.push(conntrack);
+}
+
+unsafe {
+nfct_destroy(ct);
+}
+})
}
fn insert_conntrack(&mut self, ct: Conntrack) -> Result<()> {
@@ -195,7 +190,7 @@ impl Socket {
nfct_destroy(cth);
}
-self.send_and_receive(hdr, 0, None, std::ptr::null_mut())?;
+self.send_and_receive(hdr, 0, |_| {})?;
Ok(())
}
@@ -208,7 +203,12 @@ impl Socket {
Ok(())
}
-fn query_expects_impl(&mut self, exps: &mut Vec, seq: u32, proto:
u8) -> Result<()> {
+fn query_expects_impl(
+&mut self,
+exps: &mut Vec,
+seq: u32,
+proto: u8,
+) -> Result<()> {
let mut buf = [0u8; MNL_SOCKET_DUMP_SIZE as _];
let hdr = build_msg_header(
buf.as_mut_ptr() as _,
@@ -217,7 +217,75 @@ impl Socket {
seq,
proto,
);
-self.send_and_receive(hdr, 0, Some(query_exp_cb), exps as *mut
Vec as _)
+self.send_and_receive(hdr, 0, |nlh| {
+let exp = unsafe { nfexp_new() };
+unsafe {
+nfexp_nlmsg_parse(nlh, exp);
+}
+
+let mut attributes = Vec::new();
+for (attr, ty) in EXPECT_ALL_ATTRIBUTES {
+if unsafe { nfexp_attr_is_set(exp, *attr) } == 0 {
+continue;
+}
+match ty {
+ExpectAttrType::CT => {
+let ct = unsafe { nfexp_get_attr(exp, *attr) };
+if let Some(ct) = parse_conntrack(ct as _) {
+attributes.push(ExpectAttr {
+key: *attr,
+value: ExpectAttrValue::CT(ct),
+});
+}
+}
+ExpectAttrType::U8 => {
+let val = unsafe { nfexp_get_attr_u8(exp, *attr) };
+attributes.push(ExpectAttr {
+key: *attr,
+value: ExpectAttrValue::U8(val),
+});
+}
+ExpectAttrType::U16 => {
+let val = unsafe { nfexp_get_attr_u16(exp, *attr) };
+attributes.push(ExpectAttr {
+key: *attr,
+value: ExpectAttrValue::U16(val),
+});
+}
+ExpectAttrType::U32 => {
+let val = unsafe { nfexp_get_attr_u32(exp, *attr) };
+attributes.push(ExpectAttr {
+key: *attr,
+value: ExpectAttrValue::U32(val),
+});
+}
+ExpectAttrType::String(Some(len)) => {
+let ptr = unsafe { nfexp_get_attr(exp, *attr) };
+let cstr = unsafe { std::ffi::CStr::from_ptr(ptr as _)
};
+let s = cstr.to_bytes();
+let s = unsafe {
+
CString::from_vec_unchecked(s[0..s.len().min((*len) as _)].to_vec())
+};
+attributes.push(ExpectAttr {
+key: *attr,
+value: ExpectAttrValue::String(s),
+});
+}
+ExpectAttrType::
[pve-devel] [PATCH conntrack-tool v2 2/5] add packaging support
Signed-off-by: Mira Limbeck
---
v2:
- unchanged
.cargo/config| 5
Makefile | 63
debian/changelog | 5
debian/copyright | 16 +++
debian/debcargo.toml | 17
debian/docs | 1 +
6 files changed, 107 insertions(+)
create mode 100644 .cargo/config
create mode 100644 Makefile
create mode 100644 debian/changelog
create mode 100644 debian/copyright
create mode 100644 debian/debcargo.toml
create mode 100644 debian/docs
diff --git a/.cargo/config b/.cargo/config
new file mode 100644
index 000..3b5b6e4
--- /dev/null
+++ b/.cargo/config
@@ -0,0 +1,5 @@
+[source]
+[source.debian-packages]
+directory = "/usr/share/cargo/registry"
+[source.crates-io]
+replace-with = "debian-packages"
diff --git a/Makefile b/Makefile
new file mode 100644
index 000..8ea2f8a
--- /dev/null
+++ b/Makefile
@@ -0,0 +1,63 @@
+include /usr/share/dpkg/pkg-info.mk
+include /usr/share/dpkg/architecture.mk
+
+PACKAGE=pve-conntrack-tool
+
+GITVERSION:=$(shell git rev-parse HEAD)
+
+DEB=${PACKAGE}_${DEB_VERSION_UPSTREAM_REVISION}_${DEB_BUILD_ARCH}.deb
+DSC=rust-${PACKAGE}_${DEB_VERSION_UPSTREAM_REVISION}.dsc
+
+ifeq ($(BUILD_MODE), release)
+CARGO_BUILD_ARGS += --release
+COMPILEDIR := target/release
+else
+COMPILEDIR := target/debug
+endif
+
+all: cargo-build $(SUBDIRS)
+
+.PHONY: cargo-build
+cargo-build:
+ cargo build $(CARGO_BUILD_ARGS)
+
+.PHONY: build
+build:
+ rm -rf build
+ debcargo package \
+ --config debian/debcargo.toml \
+ --changelog-ready \
+ --no-overlay-write-back \
+ --directory build \
+ $(PACKAGE) \
+ $(shell dpkg-parsechangelog -l debian/changelog -SVersion | sed -e
's/-.*//')
+ rm build/Cargo.lock
+ find build/debian -name "*.hint" -delete
+
+.PHONY: deb
+deb: $(DEB)
+$(DEB): build
+ cd build; dpkg-buildpackage -b -us -uc --no-pre-clean
--build-profiles=nodoc
+ lintian $(DEB)
+
+.PHONY: dsc
+dsc: $(DSC)
+$(DSC): build
+ cd build; dpkg-buildpackage -S -us -uc -d -nc
+ lintian $(DSC)
+
+.PHONY: dinstall
+dinstall: ${DEB}
+ dpkg -i ${DEB}
+
+.PHONY: upload
+upload: ${DEB} ${DBG_DEB}
+ tar cf - ${DEB} ${DBG_DEB}| ssh -X repo...@repo.proxmox.com -- upload
--product pve --dist buster --arch ${DEB_BUILD_ARCH}
+
+.PHONY: distclean
+distclean: clean
+
+.PHONY: clean
+clean:
+ rm -rf *.deb ${PACKAGE}-* *.buildinfo *.changes *.dsc
rust-${PACKAGE}_*.tar.?z build/
+ find . -name '*~' -exec rm {} ';'
diff --git a/debian/changelog b/debian/changelog
new file mode 100644
index 000..700e739
--- /dev/null
+++ b/debian/changelog
@@ -0,0 +1,5 @@
+rust-pve-conntrack-tool (1.0.0-1) pve; urgency=medium
+
+ * Initial release.
+
+ -- Mira Limbeck Thu, 08 Oct 2020 14:04:19 +0200
diff --git a/debian/copyright b/debian/copyright
new file mode 100644
index 000..4d6f36a
--- /dev/null
+++ b/debian/copyright
@@ -0,0 +1,16 @@
+Copyright (C) 2020 Proxmox Server Solutions GmbH
+
+This software is written by Proxmox Server Solutions GmbH
+
+This program is free software: you can redistribute it and/or modify
+it under the terms of the GNU Affero General Public License as published by
+the Free Software Foundation, either version 3 of the License, or
+(at your option) any later version.
+
+This program is distributed in the hope that it will be useful,
+but WITHOUT ANY WARRANTY; without even the implied warranty of
+MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+GNU Affero General Public License for more details.
+
+You should have received a copy of the GNU Affero General Public License
+along with this program. If not, see
[pve-devel] [PATCH qemu-server v2] copy conntrack information on migration
Requires the pve-conntrack-tool. On migration the conntrack information
from the source node is dumped and sent to the target node where it is
then inserted.
This helps with open connections during migration when the firewall is active.
A new 'migrate-conntracks' option is added to the migrate_vm API call.
Signed-off-by: Mira Limbeck
---
v2:
- added the migrate-conntracks option so that it only copies conntrack
information when requested
PVE/API2/Qemu.pm | 5 +
PVE/QemuMigrate.pm | 5 +
2 files changed, 10 insertions(+)
diff --git a/PVE/API2/Qemu.pm b/PVE/API2/Qemu.pm
index 3571f5e..8c4336b 100644
--- a/PVE/API2/Qemu.pm
+++ b/PVE/API2/Qemu.pm
@@ -3556,6 +3556,11 @@ __PACKAGE__->register_method({
minimum => '0',
default => 'migrate limit from datacenter or storage config',
},
+ 'migrate-conntracks' => {
+ description => "Migrate connection tracking info.",
+ type => 'boolean',
+ optional => 1,
+ }
},
},
returns => {
diff --git a/PVE/QemuMigrate.pm b/PVE/QemuMigrate.pm
index 5c019fc..2ccef2a 100644
--- a/PVE/QemuMigrate.pm
+++ b/PVE/QemuMigrate.pm
@@ -1087,6 +1087,11 @@ sub phase2 {
die "unable to parse migration status '$stat->{status}' -
aborting\n";
}
}
+
+if ($self->{opts}->{'migrate-conntracks'}) {
+ $self->log('info', 'copy conntrack information');
+ PVE::Tools::run_command([['/usr/bin/pve-conntrack-tool', 'dump'],
[@{$self->{rem_ssh}}, '/usr/bin/pve-conntrack-tool', 'insert']]);
+}
}
sub phase2_cleanup {
--
2.20.1
___
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
[pve-devel] [PATCH conntrack-tool v2 4/5] add additional bindings
Signed-off-by: Mira Limbeck
---
v2:
- new addition, can be ignored as it only adds batch and print function
bindings which are not used in the code currently
src/mnl.rs | 31 +--
1 file changed, 29 insertions(+), 2 deletions(-)
diff --git a/src/mnl.rs b/src/mnl.rs
index a23313e..dc897f9 100644
--- a/src/mnl.rs
+++ b/src/mnl.rs
@@ -56,6 +56,12 @@ extern "C" {
size: libc::size_t,
) -> *mut libc::c_void;
pub fn mnl_nlmsg_get_payload(nlh: *const libc::nlmsghdr) -> *mut
libc::c_void;
+pub fn mnl_nlmsg_fprintf(
+fd: *mut libc::FILE,
+data: *const libc::c_void,
+datalen: libc::size_t,
+extra_header_size: libc::size_t,
+);
pub fn mnl_cb_run(
buf: *const libc::c_void,
@@ -66,6 +72,17 @@ extern "C" {
data: *mut libc::c_void,
) -> libc::c_int;
+pub fn mnl_cb_run2(
+buf: *const libc::c_void,
+numbytes: libc::size_t,
+seq: libc::c_uint,
+portid: libc::c_uint,
+cb_data: Option,
+data: *const libc::c_void,
+cb_ctl_array: *const *const mnl_cb_t,
+cb_ctl_array_len: libc::c_uint,
+) -> libc::c_int;
+
pub fn mnl_attr_parse(
nlh: *const libc::nlmsghdr,
offset: libc::c_uint,
@@ -75,16 +92,26 @@ extern "C" {
pub fn mnl_attr_get_type(attr: *const libc::nlattr) -> u16;
pub fn mnl_attr_type_valid(attr: *const libc::nlattr, maxtype: u16) ->
libc::c_int;
-pub fn mnl_nlmsg_batch_start(buf: *mut libc::c_void, limit: libc::size_t)
-> *mut mnl_nlmsg_batch;
+pub fn mnl_nlmsg_batch_start(
+buf: *mut libc::c_void,
+limit: libc::size_t,
+) -> *mut mnl_nlmsg_batch;
pub fn mnl_nlmsg_batch_stop(b: *mut mnl_nlmsg_batch);
pub fn mnl_nlmsg_batch_next(b: *mut mnl_nlmsg_batch) -> bool;
pub fn mnl_nlmsg_batch_current(b: *mut mnl_nlmsg_batch) -> *mut
libc::c_void;
pub fn mnl_nlmsg_batch_head(b: *mut mnl_nlmsg_batch) -> *mut libc::c_void;
pub fn mnl_nlmsg_batch_size(b: *mut mnl_nlmsg_batch) -> libc::size_t;
+pub fn mnl_nlmsg_batch_reset(b: *mut mnl_nlmsg_batch);
+pub fn mnl_nlmsg_batch_is_empty(b: *mut mnl_nlmsg_batch) -> bool;
+
+pub fn mnl_nlmsg_ok(nlh: *const libc::nlmsghdr, len: libc::c_int) -> bool;
+pub fn mnl_nlmsg_next(nlh: *const libc::nlmsghdr, len: *mut libc::c_int)
+-> *mut libc::nlmsghdr;
}
#[allow(non_camel_case_types)]
-pub type mnl_cb_t = extern "C" fn(nlh: *const libc::nlmsghdr, data: *mut
libc::c_void) -> libc::c_int;
+pub type mnl_cb_t =
+extern "C" fn(nlh: *const libc::nlmsghdr, data: *mut libc::c_void) ->
libc::c_int;
#[allow(non_camel_case_types)]
pub type mnl_attr_cb_t =
extern "C" fn(attr: *const libc::nlattr, data: *mut libc::c_void) ->
libc::c_int;
--
2.20.1
___
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
[pve-devel] [PATCH manager] ui: storage: pbs: allow to set port
This patch allows to set a custom port to a PBS storage, by appending
it to the Server field (e.g. pbs.proxmox.com:443). The code was
taken from the RemoteEdit widget in the proxmox-backup repository.
While the storage config stores the port as separate line, the current
approach seems a bit more concise and does not mess up the balanced
layout.
Quickly tested on my setup.
Signed-off-by: Stoiko Ivanov
---
Thanks to Dominik for pointing me to the RemoteEdit.js!
www/manager6/storage/PBSEdit.js | 57 +++--
1 file changed, 55 insertions(+), 2 deletions(-)
diff --git a/www/manager6/storage/PBSEdit.js b/www/manager6/storage/PBSEdit.js
index 2479304a..45f35c76 100644
--- a/www/manager6/storage/PBSEdit.js
+++ b/www/manager6/storage/PBSEdit.js
@@ -463,11 +463,49 @@ Ext.define('PVE.storage.PBSInputPanel', {
me.column1 = [
{
xtype: me.isCreate ? 'textfield' : 'displayfield',
- name: 'server',
+ name: 'serverport',
value: '',
- vtype: 'DnsOrIp',
+ vtype: 'HostPort',
+ submitValue: false,
fieldLabel: gettext('Server'),
allowBlank: false,
+ listeners: {
+ change: function(field, newvalue) {
+ let server = newvalue;
+ let port;
+
+ let match = Proxmox.Utils.HostPort_match.exec(newvalue);
+ if (match === null) {
+ match =
Proxmox.Utils.HostPortBrackets_match.exec(newvalue);
+ if (match === null) {
+ match =
Proxmox.Utils.IP6_dotnotation_match.exec(newvalue);
+ }
+ }
+
+ if (match !== null) {
+ server = match[1];
+ if (match[2] !== undefined) {
+ port = match[2];
+ }
+ }
+
+
field.up('inputpanel').down('field[name=server]').setValue(server);
+
field.up('inputpanel').down('field[name=port]').setValue(port);
+ },
+ },
+ },
+ {
+ xtype: 'proxmoxtextfield',
+ hidden: true,
+ name: 'server',
+ },
+ {
+ xtype: 'proxmoxtextfield',
+ hidden: true,
+ cbind: {
+ deleteEmpty: '{!isCreate}',
+ },
+ name: 'port',
},
{
xtype: me.isCreate ? 'textfield' : 'displayfield',
@@ -522,4 +560,19 @@ Ext.define('PVE.storage.PBSInputPanel', {
me.callParent();
},
+
+setValues: function(values) {
+ let me = this;
+
+ let server = values.server;
+ if (values.port !== undefined) {
+ if (Proxmox.Utils.IP6_match.test(server)) {
+ server = `[${server}]`;
+ }
+ server += `:${values.port}`;
+ }
+ values.serverport = server;
+
+ return me.callParent([values]);
+},
});
--
2.20.1
___
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
Re: [pve-devel] [PATCH manager 2/7] ui: qemu/HardwareView: eslint: enforce "max-len" rule
On 2/3/21 10:50 AM, Aaron Lauterer wrote:
On 2/3/21 8:40 AM, Thomas Lamprecht wrote:
On 01.02.21 15:21, Aaron Lauterer wrote:
Signed-off-by: Aaron Lauterer
---
www/manager6/qemu/HardwareView.js | 4 +++-
1 file changed, 3 insertions(+), 1 deletion(-)
diff --git a/www/manager6/qemu/HardwareView.js
b/www/manager6/qemu/HardwareView.js
index 51c77246..fa72d9d3 100644
--- a/www/manager6/qemu/HardwareView.js
+++ b/www/manager6/qemu/HardwareView.js
@@ -593,7 +593,9 @@ Ext.define('PVE.qemu.HardwareView', {
var isEfi = key === 'efidisk0';
- remove_btn.setDisabled(rec.data.delete || rowdef.never_delete === true ||
(isUnusedDisk && !diskCap));
+ remove_btn.setDisabled(rec.data.delete ||
+ rowdef.never_delete === true ||
+ (isUnusedDisk && !diskCap));
If a method call is split over multiple lines the first line should only
be the method itself.
As we have an expression here, not really multiple parameters, either of the
following two would be fine:
remove_btn.setDisabled(
rec.data.delete || rowdef.never_delete === true || (isUnusedDisk &&
!diskCap)
);
or:
remove_btn.setDisabled(
rec.data.delete ||
rowdef.never_delete === true ||
(isUnusedDisk && !diskCap)
);
Maybe we want to add a eslint rule to catch these? I am usually getting this
wrong and having a linter rule will help to catch it early on. AFAICT the
'function-paren-newline' set to 'multiline' or 'multiline-arguments' [0] should
work.
[0] https://eslint.org/docs/rules/function-paren-newline
Of course, this won't work in this use case as they are not parameters. I'll
keep looking if there is a rule for this situation that could help.
remove_btn.setText(isUsedDisk && !isCloudInit ? remove_btn.altText :
remove_btn.defaultText);
remove_btn.RESTMethod = isUnusedDisk ? 'POST':'PUT';
___
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
___
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
Re: [pve-devel] [PATCH manager 2/7] ui: qemu/HardwareView: eslint: enforce "max-len" rule
On 2/3/21 8:40 AM, Thomas Lamprecht wrote:
On 01.02.21 15:21, Aaron Lauterer wrote:
Signed-off-by: Aaron Lauterer
---
www/manager6/qemu/HardwareView.js | 4 +++-
1 file changed, 3 insertions(+), 1 deletion(-)
diff --git a/www/manager6/qemu/HardwareView.js
b/www/manager6/qemu/HardwareView.js
index 51c77246..fa72d9d3 100644
--- a/www/manager6/qemu/HardwareView.js
+++ b/www/manager6/qemu/HardwareView.js
@@ -593,7 +593,9 @@ Ext.define('PVE.qemu.HardwareView', {
var isEfi = key === 'efidisk0';
- remove_btn.setDisabled(rec.data.delete || rowdef.never_delete === true || (isUnusedDisk && !diskCap));
+ remove_btn.setDisabled(rec.data.delete ||
+ rowdef.never_delete === true ||
+ (isUnusedDisk && !diskCap));
If a method call is split over multiple lines the first line should only
be the method itself.
As we have an expression here, not really multiple parameters, either of the
following two would be fine:
remove_btn.setDisabled(
rec.data.delete || rowdef.never_delete === true || (isUnusedDisk &&
!diskCap)
);
or:
remove_btn.setDisabled(
rec.data.delete ||
rowdef.never_delete === true ||
(isUnusedDisk && !diskCap)
);
Maybe we want to add a eslint rule to catch these? I am usually getting this
wrong and having a linter rule will help to catch it early on. AFAICT the
'function-paren-newline' set to 'multiline' or 'multiline-arguments' [0] should
work.
[0] https://eslint.org/docs/rules/function-paren-newline
remove_btn.setText(isUsedDisk && !isCloudInit ? remove_btn.altText
: remove_btn.defaultText);
remove_btn.RESTMethod = isUnusedDisk ? 'POST':'PUT';
___
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
Re: [pve-devel] [PATCH qemu-server] cloudinit: add sshdeletehostkeys option
Le lundi 01 février 2021 à 17:12 +0100, aderum...@odiso.com a écrit :
> >
> > [0] https://cloudinit.readthedocs.io/en/latest/topics/cli.html#clea
> > n
>
>
> The main problem currently is indeed that we change instance-id at
> each rebuild of the cloud-init disk.
> But I'm not sure that's it's possible to change ip address when
> keeping same instance-id, because ip configuration is done
> at the cloudinit-init-local service, at it's already done once.
> Maybe this was the historic reason why we change the the instance-id
> each time, I don't remember exactly.
> I'll check that tomorrow to be sure, but indeed, keeping the
> instance-id should be the clean way.
Hi, I have redone tests with cloud-init.
So, the problem with networking config, is it's really only done
once by instance-id , and they are no way to change that.
(it's done in cloud-init local, and it's not possible to change that
like the others modules).
So, we really need to change instance-id each time.
Also, the ssh module manage both user ssh public keys,
and host private keys. So the only way is the trick of this patch to
disable the host key generation after first boot.
Currently I'm not using cloud-init, but I wonder if it's really
the good tool to manage configuration after the first boot.
(It's has been created by cloud-provider to really only run once at
first boot, and after that the configuration don't change).
I have looked on the net, and found than opennebula provide a nice
tool to manage configuration at boot, but also changes in live.
https://github.com/OpenNebula/addon-context-linux
It's only simple a simple iso config drive with environment variables +
bash script + service for boot + udev rules to handle cdrom media
changes/nic plug/unplug live events.
Historically, opennebula was using cloud-init too (they are also an
opennebula config drive driver in cloud-init)
https://cloudinit.readthedocs.io/en/latest/topics/datasources/opennebula.html
but it was not matching their workload, so they have created their own
script.
https://forum.opennebula.io/t/one-context-vs-cloud-init/1641
I have tested it, with a small patch in CloudInit.pm to generate the
cloudera context format, it's just working out of the box.
I can manage ips,hostname,sshkeys,resize partition dynamically without
any trick.
I really like it, and with simple bash script + env var in config
drive, it's possible to easily adapt them for special need.
(I have some tricky network config in my production network)
a windows version is available too:
https://github.com/OpenNebula/addon-context-windows
What do you think to add this new datasource format support in current
Cloudinit.pm ?
Here a sample (not cleaned yet):
+sub generate_opennebula {
+my ($conf, $vmid, $drive, $volname, $storeid) = @_;
+
+my ($hostname, $fqdn) = get_hostname_fqdn($conf, $vmid);
+
+my $content = "";
+
+my $username = $conf->{ciuser};
+my $password = encode_base64($conf->{cipassword});
+$keys = [map { my $key = $_; chomp $key; $key } split(/\n/,
$keys)];
+$keys = [grep { /\S/ } @$keys];
+$content .= "SSH_PUBLIC_KEY=\"";
+
+foreach my $k (@$keys) {
+$content .= "$k\n";
+}
+$content .= "\"\n";
+
+}
+
+my ($searchdomains, $nameservers) = get_dns_conf($conf);
+if ($nameservers && @$nameservers) {
+$nameservers = join(' ', @$nameservers);
+$content .= "DNS=\"$nameservers\"\n";
+}
+
+$content .= "NETWORK=YES\n";
+$content .= "SET_HOSTNAME=prout2\n";
+
+if ($searchdomains && @$searchdomains) {
+$searchdomains = join(' ', @$searchdomains);
+$content .= "SEARCH_DOMAIN=\"$searchdomains\"\n";
+}
+
+my @ifaces = grep { /^net(\d+)$/ } keys %$conf;
+foreach my $iface (sort @ifaces) {
+(my $id = $iface) =~ s/^net//;
+next if !$conf->{"ipconfig$id"};
+my $net = PVE::QemuServer::parse_ipconfig($conf-
>{"ipconfig$id"});
+my $ethid = "ETH$id";
+
+ my $mac = lc $net->{hwaddr};
+if ($net->{ip}) {
+if ($net->{ip} eq 'dhcp') {
+$content .= "\n"; #opennebule don't handle DHCP
config.
+} else {
+my ($addr, $mask) = split_ip4($net->{ip});
+ $content .= $ethid."_IP=$addr\n";
+ $content .= $ethid."_MASK=$mask\n";
+ $content .= $ethid."_MAC=$mac\n";
+ $content .= $ethid."_GATEWAY=$net->{gw}\n" if $net-
>{gw};
+}
+}
+
+}
+
+my $files = {
+ '/context.sh' => $content,
+};
+commit_cloudinit_disk($conf, $vmid, $drive, $volname, $storeid,
$files, 'CONTEXT');
+}
+
@@ -461,13 +531,14 @@ sub read_cloudinit_snippets_file {
my $cloudinit_methods = {
configdrive2 => \&generate_configdrive2,
nocloud => \&generate_nocloud,
+opennebula => \&generate_opennebula,
};
___
pve-devel mailing list
pve-
