Re: [Devel] [PATCH] zdtm: fix package memory allocation in autofs.c
No, we don't.
But this one could be applied: "tests: do not try to read more than packet in
AutoFS test"
09.09.2017 02:01, Andrei Vagin пишет:
> On Thu, Aug 31, 2017 at 12:10:40PM +0300, Stanislav Kinsburskiy wrote:
>> Plus some cleanup.
>
> Do we need this patch for the upstream criu? Could you send it to
> the criu mailing list?
>
>>
>> https://jira.sw.ru/browse/PSBM-71078
>>
>> Signed-off-by: Stanislav Kinsburskiy
>> ---
>> test/zdtm/static/autofs.c | 18 +-
>> 1 file changed, 9 insertions(+), 9 deletions(-)
>>
>> diff --git a/test/zdtm/static/autofs.c b/test/zdtm/static/autofs.c
>> index 8d917ee..882289f 100644
>> --- a/test/zdtm/static/autofs.c
>> +++ b/test/zdtm/static/autofs.c
>> @@ -460,10 +460,10 @@ static int automountd_loop(int pipe, const char
>> *mountpoint, struct autofs_param
>> {
>> union autofs_v5_packet_union *packet;
>> ssize_t bytes;
>> -size_t psize = sizeof(*packet) * 2;
>> +size_t psize = sizeof(*packet);
>> int err = 0;
>>
>> -packet = malloc(psize);
>> +packet = malloc(psize * 2);
>> if (!packet) {
>> pr_err("failed to allocate autofs packet\n");
>> return -ENOMEM;
>> @@ -473,7 +473,7 @@ static int automountd_loop(int pipe, const char
>> *mountpoint, struct autofs_param
>> siginterrupt(SIGUSR2, 1);
>>
>> while (!stop && !err) {
>> -memset(packet, 0, sizeof(*packet));
>> +memset(packet, 0, psize * 2);
>>
>> bytes = read(pipe, packet, psize);
>> if (bytes < 0) {
>> @@ -483,12 +483,12 @@ static int automountd_loop(int pipe, const char
>> *mountpoint, struct autofs_param
>> }
>> continue;
>> }
>> -if (bytes > psize) {
>> -pr_err("read more that expected: %zd > %zd\n", bytes,
>> psize);
>> -return -EINVAL;
>> -}
>> -if (bytes != sizeof(*packet)) {
>> -pr_err("read less than expected: %zd\n", bytes);
>> +if (bytes != psize) {
>> +pr_err("read %s that expected: %zd %s %zd\n",
>> +(bytes > psize) ? "more" : "less",
>> +bytes,
>> +(bytes > psize) ? ">" : "<",
>> +psize);
>> return -EINVAL;
>> }
>> err = automountd_serve(mountpoint, param, packet);
>>
>> ___
>> Devel mailing list
>> Devel@openvz.org
>> https://lists.openvz.org/mailman/listinfo/devel
___
Devel mailing list
Devel@openvz.org
https://lists.openvz.org/mailman/listinfo/devel
[Devel] [PATCH rh7 1/4] fs/ext4: __ext4_es_shrink() rename 'ret' to 'shrunk'
This renames 'ret' variable to 'shrunk' as it used to count
shrunk extents.
This patch is the lost hunk from the upstream version of
patch "fs: convert fs shrinkers to new scan/count API",
which doesn't present in our backport:
9d22f7aba1b7ee7014fadd042a49be27a308ff74
Author: Vladimir Davydov
Date: Fri Jul 17 11:51:54 2015 +0300
ms/fs: convert fs shrinkers to new scan/count API
Signed-off-by: Andrey Ryabinin
---
fs/ext4/extents_status.c | 10 ++
1 file changed, 6 insertions(+), 4 deletions(-)
diff --git a/fs/ext4/extents_status.c b/fs/ext4/extents_status.c
index 71a6e72e70aa..6f1f614c12ce 100644
--- a/fs/ext4/extents_status.c
+++ b/fs/ext4/extents_status.c
@@ -943,13 +943,15 @@ static int __ext4_es_shrink(struct ext4_sb_info *sbi, int
nr_to_scan,
struct ext4_inode_info *ei;
struct list_head *cur, *tmp;
LIST_HEAD(skipped);
- int ret, nr_shrunk = 0;
+ int nr_shrunk = 0;
int retried = 0, skip_precached = 1, nr_skipped = 0;
spin_lock(&sbi->s_es_lru_lock);
retry:
list_for_each_safe(cur, tmp, &sbi->s_es_lru) {
+ int shrunk;
+
/*
* If we have already reclaimed all extents from extent
* status tree, just stop the loop immediately.
@@ -976,13 +978,13 @@ retry:
!write_trylock(&ei->i_es_lock))
continue;
- ret = __es_try_to_reclaim_extents(ei, nr_to_scan);
+ shrunk = __es_try_to_reclaim_extents(ei, nr_to_scan);
if (ei->i_es_lru_nr == 0)
list_del_init(&ei->i_es_lru);
write_unlock(&ei->i_es_lock);
- nr_shrunk += ret;
- nr_to_scan -= ret;
+ nr_shrunk += shrunk;
+ nr_to_scan -= shrunk;
if (nr_to_scan == 0)
break;
}
--
2.13.5
___
Devel mailing list
Devel@openvz.org
https://lists.openvz.org/mailman/listinfo/devel
[Devel] [PATCH rh7 4/4] ext4: change LRU to round-robin in extent status tree shrinker
From: Zheng Liu
commit edaa53cac8fd4b96ed4b8f96c4933158ff2dd337 upstream.
In this commit we discard the lru algorithm for inodes with extent
status tree because it takes significant effort to maintain a lru list
in extent status tree shrinker and the shrinker can take a long time to
scan this lru list in order to reclaim some objects.
We replace the lru ordering with a simple round-robin. After that we
never need to keep a lru list. That means that the list needn't be
sorted if the shrinker can not reclaim any objects in the first round.
Cc: Andreas Dilger
Signed-off-by: Zheng Liu
Signed-off-by: Jan Kara
Signed-off-by: Theodore Ts'o
https://jira.sw.ru/browse/PSBM-70858
Signed-off-by: Andrey Ryabinin
---
fs/ext4/ext4.h | 10 +-
fs/ext4/extents.c | 4 +-
fs/ext4/extents_status.c| 224 +++-
fs/ext4/extents_status.h| 7 +-
fs/ext4/inode.c | 4 +-
fs/ext4/ioctl.c | 4 +-
fs/ext4/super.c | 7 +-
include/trace/events/ext4.h | 11 +--
8 files changed, 118 insertions(+), 153 deletions(-)
diff --git a/fs/ext4/ext4.h b/fs/ext4/ext4.h
index 5ff07c5eaaa2..2ad7a2953de0 100644
--- a/fs/ext4/ext4.h
+++ b/fs/ext4/ext4.h
@@ -905,10 +905,9 @@ struct ext4_inode_info {
/* extents status tree */
struct ext4_es_tree i_es_tree;
rwlock_t i_es_lock;
- struct list_head i_es_lru;
+ struct list_head i_es_list;
unsigned int i_es_all_nr; /* protected by i_es_lock */
- unsigned int i_es_lru_nr; /* protected by i_es_lock */
- unsigned long i_touch_when; /* jiffies of last accessing */
+ unsigned int i_es_shk_nr; /* protected by i_es_lock */
/* ialloc */
ext4_group_ti_last_alloc_group;
@@ -1363,9 +1362,10 @@ struct ext4_sb_info {
/* Reclaim extents from extent status tree */
struct shrinker s_es_shrinker;
- struct list_head s_es_lru;
+ struct list_head s_es_list;
+ long s_es_nr_inode;
struct ext4_es_stats s_es_stats;
- spinlock_t s_es_lru_lock cacheline_aligned_in_smp;
+ spinlock_t s_es_lock cacheline_aligned_in_smp;
/* Ratelimit ext4 messages. */
struct ratelimit_state s_err_ratelimit_state;
diff --git a/fs/ext4/extents.c b/fs/ext4/extents.c
index 0f7c73f168a5..4ed1c1a703a0 100644
--- a/fs/ext4/extents.c
+++ b/fs/ext4/extents.c
@@ -4653,7 +4653,7 @@ out2:
trace_ext4_ext_map_blocks_exit(inode, flags, map,
err ? err : allocated);
- ext4_es_lru_add(inode);
+ ext4_es_list_add(inode);
return err ? err : allocated;
}
@@ -5242,7 +5242,7 @@ int ext4_fiemap(struct inode *inode, struct
fiemap_extent_info *fieinfo,
error = ext4_fill_fiemap_extents(inode, start_blk,
len_blks, fieinfo);
}
- ext4_es_lru_add(inode);
+ ext4_es_list_add(inode);
return error;
}
diff --git a/fs/ext4/extents_status.c b/fs/ext4/extents_status.c
index c75ebf0a3356..b812002db5e2 100644
--- a/fs/ext4/extents_status.c
+++ b/fs/ext4/extents_status.c
@@ -149,8 +149,8 @@ static int __es_remove_extent(struct inode *inode,
ext4_lblk_t lblk,
ext4_lblk_t end);
static int __es_try_to_reclaim_extents(struct ext4_inode_info *ei,
int nr_to_scan);
-static int __ext4_es_shrink(struct ext4_sb_info *sbi, int nr_to_scan,
- struct ext4_inode_info *locked_ei);
+static int __es_shrink(struct ext4_sb_info *sbi, int nr_to_scan,
+ struct ext4_inode_info *locked_ei);
int __init ext4_init_es(void)
{
@@ -298,6 +298,36 @@ out:
trace_ext4_es_find_delayed_extent_range_exit(inode, es);
}
+void ext4_es_list_add(struct inode *inode)
+{
+ struct ext4_inode_info *ei = EXT4_I(inode);
+ struct ext4_sb_info *sbi = EXT4_SB(inode->i_sb);
+
+ if (!list_empty(&ei->i_es_list))
+ return;
+
+ spin_lock(&sbi->s_es_lock);
+ if (list_empty(&ei->i_es_list)) {
+ list_add_tail(&ei->i_es_list, &sbi->s_es_list);
+ sbi->s_es_nr_inode++;
+ }
+ spin_unlock(&sbi->s_es_lock);
+}
+
+void ext4_es_list_del(struct inode *inode)
+{
+ struct ext4_inode_info *ei = EXT4_I(inode);
+ struct ext4_sb_info *sbi = EXT4_SB(inode->i_sb);
+
+ spin_lock(&sbi->s_es_lock);
+ if (!list_empty(&ei->i_es_list)) {
+ list_del_init(&ei->i_es_list);
+ sbi->s_es_nr_inode--;
+ WARN_ON_ONCE(sbi->s_es_nr_inode < 0);
+ }
+ spin_unlock(&sbi->s_es_lock);
+}
+
static struct extent_status *
ext4_es_alloc_extent(struct inode *inode, ext4_lblk_t lblk, ext4_lblk_t len,
ext4_fsblk_t pblk)
@@ -314,9 +344,9 @@ ext4_es_alloc_extent(struct inode *inode, ext4_lblk_t lblk,
ext4_lblk_t l
[Devel] [PATCH rh7 2/4] ext4: improve extents status tree trace point
From: Zheng Liu
commit e963bb1de415ab06693357336c1bec664753e1e2 upsteam.
This commit improves the trace point of extents status tree. We rename
trace_ext4_es_shrink_enter in ext4_es_count() because it is also used
in ext4_es_scan() and we can not identify them from the result.
Further this commit fixes a variable name in trace point in order to
keep consistency with others.
Cc: Andreas Dilger
Cc: Jan Kara
Reviewed-by: Jan Kara
Signed-off-by: Zheng Liu
Signed-off-by: Theodore Ts'o
https://jira.sw.ru/browse/PSBM-70858
Signed-off-by: Andrey Ryabinin
---
fs/ext4/extents_status.c| 6 +++---
include/trace/events/ext4.h | 28
2 files changed, 23 insertions(+), 11 deletions(-)
diff --git a/fs/ext4/extents_status.c b/fs/ext4/extents_status.c
index 6f1f614c12ce..50b776a794a5 100644
--- a/fs/ext4/extents_status.c
+++ b/fs/ext4/extents_status.c
@@ -1029,7 +1029,7 @@ static unsigned long ext4_es_count(struct shrinker
*shrink,
sbi = container_of(shrink, struct ext4_sb_info, s_es_shrinker);
nr = percpu_counter_read_positive(&sbi->s_extent_cache_cnt);
- trace_ext4_es_shrink_enter(sbi->s_sb, sc->nr_to_scan, nr);
+ trace_ext4_es_shrink_count(sbi->s_sb, sc->nr_to_scan, nr);
return nr;
}
@@ -1042,14 +1042,14 @@ static unsigned long ext4_es_scan(struct shrinker
*shrink,
int ret, nr_shrunk;
ret = percpu_counter_read_positive(&sbi->s_extent_cache_cnt);
- trace_ext4_es_shrink_enter(sbi->s_sb, nr_to_scan, ret);
+ trace_ext4_es_shrink_scan_enter(sbi->s_sb, nr_to_scan, ret);
if (!nr_to_scan)
return ret;
nr_shrunk = __ext4_es_shrink(sbi, nr_to_scan, NULL);
- trace_ext4_es_shrink_exit(sbi->s_sb, nr_shrunk, ret);
+ trace_ext4_es_shrink_scan_exit(sbi->s_sb, nr_shrunk, ret);
return nr_shrunk;
}
diff --git a/include/trace/events/ext4.h b/include/trace/events/ext4.h
index 68adee93392f..316ccd579d43 100644
--- a/include/trace/events/ext4.h
+++ b/include/trace/events/ext4.h
@@ -2461,7 +2461,7 @@ TRACE_EVENT(ext4_es_lookup_extent_exit,
show_extent_status(__entry->found ? __entry->status : 0))
);
-TRACE_EVENT(ext4_es_shrink_enter,
+DECLARE_EVENT_CLASS(ext4__es_shrink_enter,
TP_PROTO(struct super_block *sb, int nr_to_scan, int cache_cnt),
TP_ARGS(sb, nr_to_scan, cache_cnt),
@@ -2483,26 +2483,38 @@ TRACE_EVENT(ext4_es_shrink_enter,
__entry->nr_to_scan, __entry->cache_cnt)
);
-TRACE_EVENT(ext4_es_shrink_exit,
- TP_PROTO(struct super_block *sb, int shrunk_nr, int cache_cnt),
+DEFINE_EVENT(ext4__es_shrink_enter, ext4_es_shrink_count,
+ TP_PROTO(struct super_block *sb, int nr_to_scan, int cache_cnt),
+
+ TP_ARGS(sb, nr_to_scan, cache_cnt)
+);
+
+DEFINE_EVENT(ext4__es_shrink_enter, ext4_es_shrink_scan_enter,
+ TP_PROTO(struct super_block *sb, int nr_to_scan, int cache_cnt),
+
+ TP_ARGS(sb, nr_to_scan, cache_cnt)
+);
+
+TRACE_EVENT(ext4_es_shrink_scan_exit,
+ TP_PROTO(struct super_block *sb, int nr_shrunk, int cache_cnt),
- TP_ARGS(sb, shrunk_nr, cache_cnt),
+ TP_ARGS(sb, nr_shrunk, cache_cnt),
TP_STRUCT__entry(
__field(dev_t, dev )
- __field(int,shrunk_nr )
+ __field(int,nr_shrunk )
__field(int,cache_cnt )
),
TP_fast_assign(
__entry->dev= sb->s_dev;
- __entry->shrunk_nr = shrunk_nr;
+ __entry->nr_shrunk = nr_shrunk;
__entry->cache_cnt = cache_cnt;
),
- TP_printk("dev %d,%d shrunk_nr %d cache_cnt %d",
+ TP_printk("dev %d,%d nr_shrunk %d cache_cnt %d",
MAJOR(__entry->dev), MINOR(__entry->dev),
- __entry->shrunk_nr, __entry->cache_cnt)
+ __entry->nr_shrunk, __entry->cache_cnt)
);
TRACE_EVENT(ext4_collapse_range,
--
2.13.5
___
Devel mailing list
Devel@openvz.org
https://lists.openvz.org/mailman/listinfo/devel
[Devel] [PATCH rh7 3/4] ext4: track extent status tree shrinker delay statictics
From: Zheng Liu
commit eb68d0e2fc5a4e5c06324ea5f485fccbae626d05 upstream.
This commit adds some statictics in extent status tree shrinker. The
purpose to add these is that we want to collect more details when we
encounter a stall caused by extent status tree shrinker. Here we count
the following statictics:
stats:
the number of all objects on all extent status trees
the number of reclaimable objects on lru list
cache hits/misses
the last sorted interval
the number of inodes on lru list
average:
scan time for shrinking some objects
the number of shrunk objects
maximum:
the inode that has max nr. of objects on lru list
the maximum scan time for shrinking some objects
The output looks like below:
$ cat /proc/fs/ext4/sda1/es_shrinker_info
stats:
28228 objects
6341 reclaimable objects
5281/631 cache hits/misses
586 ms last sorted interval
250 inodes on lru list
average:
153 us scan time
128 shrunk objects
maximum:
255 inode (255 objects, 198 reclaimable)
125723 us max scan time
If the lru list has never been sorted, the following line will not be
printed:
586ms last sorted interval
If there is an empty lru list, the following lines also will not be
printed:
250 inodes on lru list
...
maximum:
255 inode (255 objects, 198 reclaimable)
0 us max scan time
Meanwhile in this commit a new trace point is defined to print some
details in __ext4_es_shrink().
Cc: Andreas Dilger
Cc: Jan Kara
Reviewed-by: Jan Kara
Signed-off-by: Zheng Liu
Signed-off-by: Theodore Ts'o
https://jira.sw.ru/browse/PSBM-70858
Signed-off-by: Andrey Ryabinin
---
fs/ext4/ext4.h | 4 +-
fs/ext4/extents_status.c| 186 +---
fs/ext4/extents_status.h| 13 +++-
fs/ext4/super.c | 17 ++--
include/trace/events/ext4.h | 31
5 files changed, 227 insertions(+), 24 deletions(-)
diff --git a/fs/ext4/ext4.h b/fs/ext4/ext4.h
index d5a438632ada..5ff07c5eaaa2 100644
--- a/fs/ext4/ext4.h
+++ b/fs/ext4/ext4.h
@@ -906,6 +906,7 @@ struct ext4_inode_info {
struct ext4_es_tree i_es_tree;
rwlock_t i_es_lock;
struct list_head i_es_lru;
+ unsigned int i_es_all_nr; /* protected by i_es_lock */
unsigned int i_es_lru_nr; /* protected by i_es_lock */
unsigned long i_touch_when; /* jiffies of last accessing */
@@ -1363,8 +1364,7 @@ struct ext4_sb_info {
/* Reclaim extents from extent status tree */
struct shrinker s_es_shrinker;
struct list_head s_es_lru;
- unsigned long s_es_last_sorted;
- struct percpu_counter s_extent_cache_cnt;
+ struct ext4_es_stats s_es_stats;
spinlock_t s_es_lru_lock cacheline_aligned_in_smp;
/* Ratelimit ext4 messages. */
diff --git a/fs/ext4/extents_status.c b/fs/ext4/extents_status.c
index 50b776a794a5..c75ebf0a3356 100644
--- a/fs/ext4/extents_status.c
+++ b/fs/ext4/extents_status.c
@@ -11,6 +11,8 @@
*/
#include
#include
+#include
+#include
#include "ext4.h"
#include "extents_status.h"
@@ -313,19 +315,27 @@ ext4_es_alloc_extent(struct inode *inode, ext4_lblk_t
lblk, ext4_lblk_t len,
*/
if (!ext4_es_is_delayed(es)) {
EXT4_I(inode)->i_es_lru_nr++;
- percpu_counter_inc(&EXT4_SB(inode->i_sb)->s_extent_cache_cnt);
+ percpu_counter_inc(&EXT4_SB(inode->i_sb)->
+ s_es_stats.es_stats_lru_cnt);
}
+ EXT4_I(inode)->i_es_all_nr++;
+ percpu_counter_inc(&EXT4_SB(inode->i_sb)->s_es_stats.es_stats_all_cnt);
+
return es;
}
static void ext4_es_free_extent(struct inode *inode, struct extent_status *es)
{
+ EXT4_I(inode)->i_es_all_nr--;
+ percpu_counter_dec(&EXT4_SB(inode->i_sb)->s_es_stats.es_stats_all_cnt);
+
/* Decrease the lru counter when this es is not delayed */
if (!ext4_es_is_delayed(es)) {
BUG_ON(EXT4_I(inode)->i_es_lru_nr == 0);
EXT4_I(inode)->i_es_lru_nr--;
- percpu_counter_dec(&EXT4_SB(inode->i_sb)->s_extent_cache_cnt);
+ percpu_counter_dec(&EXT4_SB(inode->i_sb)->
+ s_es_stats.es_stats_lru_cnt);
}
kmem_cache_free(ext4_es_cachep, es);
@@ -739,6 +749,7 @@ int ext4_es_lookup_extent(struct inode *inode, ext4_lblk_t
lblk,
struct extent_status *es)
{
struct ext4_es_tree *tree;
+ struct ext4_es_stats *stats;
struct extent_status *es1 = NULL;
struct rb_node *node;
int found = 0;
@@ -775,11 +786,15 @@ int ext4_es_lookup_extent(struct inode *inode,
ext4_lblk_t lblk,
}
out:
+ stats = &EXT4_SB(inode->i_sb)->s_es_stats;
if (found) {
BUG_ON(!es1);
es->es_lblk = es1->es_lblk;
es->es_len = es1->es_len;
es
Re: [Devel] [PATCH] criu: threads -- Allow uids/gids being different
On Sun, Jul 23, 2017 at 6:15 PM, Cyrill Gorcunov wrote: > We are carrying uids/gids in per-thread way since v1.8, > so no need to require them to match anymore. We still > require seccomp to match though. Need to revisit, seems > may relax this requirement as well. > > https://jira.sw.ru/browse/PSBM-68756 > > Signed-off-by: Cyrill Gorcunov Ping? ___ Devel mailing list Devel@openvz.org https://lists.openvz.org/mailman/listinfo/devel
Re: [Devel] [PATCH] criu: threads -- Allow uids/gids being different
On Tue, Sep 12, 2017 at 5:43 PM, Cyrill Gorcunov wrote: > On Sun, Jul 23, 2017 at 6:15 PM, Cyrill Gorcunov wrote: >> We are carrying uids/gids in per-thread way since v1.8, >> so no need to require them to match anymore. We still >> require seccomp to match though. Need to revisit, seems >> may relax this requirement as well. >> >> https://jira.sw.ru/browse/PSBM-68756 >> >> Signed-off-by: Cyrill Gorcunov > > Ping? Drop it, I resent new version with testcase from Vitaly. ___ Devel mailing list Devel@openvz.org https://lists.openvz.org/mailman/listinfo/devel
[Devel] [PATCH] sysfs: don't use VE namespace for "dev/char" directory
Looks like we don't have devices, using it. Or lost some patches.
Anyway, currently it doesn't work. To make this VE namespace work, "namespace"
callback has to be added to ve_kobj_type. And this callback has to get VE
structure from kobject somehow. IOW, device has to be allocated per-ve.
Another issue here is that it's unclear, how to mark generic devices as
allocated in VE#0 thus making them visible in VE#0.
Looks like the whole idea was wrong...
https://jira.sw.ru/browse/PSBM-71811
Signed-off-by: Stanislav Kinsburskiy
---
drivers/base/core.c |2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/drivers/base/core.c b/drivers/base/core.c
index 80cf3eb..86540d9 100644
--- a/drivers/base/core.c
+++ b/drivers/base/core.c
@@ -1538,7 +1538,7 @@ int __init devices_init(void)
sysfs_dev_block_kobj = kobject_create_and_add("block", dev_kobj);
if (!sysfs_dev_block_kobj)
goto block_kobj_err;
- sysfs_dev_char_kobj = kobject_create_and_add_ve("char", dev_kobj);
+ sysfs_dev_char_kobj = kobject_create_and_add("char", dev_kobj);
if (!sysfs_dev_char_kobj)
goto char_kobj_err;
___
Devel mailing list
Devel@openvz.org
https://lists.openvz.org/mailman/listinfo/devel
