[PATCH] opensm/osm_lid_mgr: use 'first_time_master_sweep' flag
Use 'first_time_master_sweep' as initial sweep indication instead of
'coming_out_of_standby'. The last flag will be set only when OpenSM
previously was in standby state, this doesn't cover some other cases
such as temporary SM port disconnection.
Signed-off-by: Sasha Khapyorsky sas...@voltaire.com
---
opensm/opensm/osm_lid_mgr.c |2 +-
1 files changed, 1 insertions(+), 1 deletions(-)
diff --git a/opensm/opensm/osm_lid_mgr.c b/opensm/opensm/osm_lid_mgr.c
index 0fd3b20..84f3508 100644
--- a/opensm/opensm/osm_lid_mgr.c
+++ b/opensm/opensm/osm_lid_mgr.c
@@ -304,7 +304,7 @@ static int lid_mgr_init_sweep(IN osm_lid_mgr_t * p_mgr)
info we might have.
Do this only if the honor_guid2lid_file option is FALSE. If not, then
need to honor this file. */
- if (p_mgr-p_subn-coming_out_of_standby == TRUE) {
+ if (p_mgr-p_subn-first_time_master_sweep == TRUE) {
osm_db_clear(p_mgr-p_g2l);
if (p_mgr-p_subn-opt.honor_guid2lid_file == FALSE) {
OSM_LOG(p_mgr-p_log, OSM_LOG_DEBUG,
--
1.7.0.4
--
To unsubscribe from this list: send the line unsubscribe linux-rdma in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH] opensm/osm_lid_mgr: fix couple of duplicate LIDs bugs
Fix couple of cases where duplicated LID generation is possible: 1) When guid2lid db is reloaded after standby or disconnected SM port state (discovering) it must be validated, so content of used_lids[] array (used as reference during LID assignment) will be consistent with loaded guid2lid db. 2) When port for which LIDs were assigned exists in guid2lid db already its record should be updated due to possibly changed LIDs, LMC, etc.. There still be more potential cases where LIDs could be duplicated. So this patch fixes just part of them. I'm continuing an investigation. Signed-off-by: Sasha Khapyorsky sas...@voltaire.com --- opensm/opensm/osm_lid_mgr.c |4 ++-- 1 files changed, 2 insertions(+), 2 deletions(-) diff --git a/opensm/opensm/osm_lid_mgr.c b/opensm/opensm/osm_lid_mgr.c index 84f3508..08d54f0 100644 --- a/opensm/opensm/osm_lid_mgr.c +++ b/opensm/opensm/osm_lid_mgr.c @@ -319,6 +319,7 @@ static int lid_mgr_init_sweep(IN osm_lid_mgr_t * p_mgr) ERR 0306: Error restoring Guid-to-Lid persistent database. Ignoring it\n); + lid_mgr_validate_db(p_mgr); } } @@ -710,7 +711,7 @@ static int lid_mgr_get_port_lid(IN osm_lid_mgr_t * p_mgr, /* we still need to send the setting to the target port */ lid_changed = 1; } - goto Exit; + goto NewLidSet; } else OSM_LOG(p_mgr-p_log, OSM_LOG_DEBUG, 0x%016 PRIx64 has no persistent lid assigned\n, @@ -764,7 +765,6 @@ NewLidSet: for (lid = *p_min_lid; lid = *p_max_lid; lid++) p_mgr-used_lids[lid] = 1; -Exit: /* make sure the assigned lids are marked in port_lid_tbl */ for (lid = *p_min_lid; lid = *p_max_lid; lid++) cl_ptr_vector_set(p_mgr-p_subn-port_lid_tbl, lid, p_port); -- 1.7.0.4 -- To unsubscribe from this list: send the line unsubscribe linux-rdma in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH v3 05/10] iw_cxgb4: Add connection management functions.
+int c4iw_ep_redirect(void *ctx, struct dst_entry *old, struct dst_entry
*new,
As far as I can see this function is not called or otherwise referenced
anywhere else (except for a declaration in a header). Can we drop it?
Yes.
OK, I rolled the following up into the big merge iw_cxgb4 patch I have queued:
diff --git a/drivers/infiniband/hw/cxgb4/cm.c b/drivers/infiniband/hw/cxgb4/cm.c
index e1c1ee0..b602438 100644
--- a/drivers/infiniband/hw/cxgb4/cm.c
+++ b/drivers/infiniband/hw/cxgb4/cm.c
@@ -2260,24 +2260,6 @@ int c4iw_ep_disconnect(struct c4iw_ep *ep, int abrupt,
gfp_t gfp)
return ret;
}
-int c4iw_ep_redirect(void *ctx, struct dst_entry *old, struct dst_entry *new,
-struct l2t_entry *l2t)
-{
- struct c4iw_ep *ep = ctx;
-
- if (ep-dst != old)
- return 0;
-
- PDBG(%s ep %p redirect to dst %p l2t %p\n, __func__, ep, new,
-l2t);
- dst_hold(new);
- cxgb4_l2t_release(ep-l2t);
- ep-l2t = l2t;
- dst_release(old);
- ep-dst = new;
- return 1;
-}
-
/*
* All the CM events are handled on a work queue to have a safe context.
*/
--
Roland Dreier rola...@cisco.com || For corporate legal information go to:
http://www.cisco.com/web/about/doing_business/legal/cri/index.html
--
To unsubscribe from this list: send the line unsubscribe linux-rdma in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: SDP bugs 2027 and 2028
Amir Vadai wrote: I have a fix for 2027 (number of SGE's issue) read. I'm busy with other things - but hopefully will succeed testing it and pushing it today. As to 2028 (No FMR support) - I need pushed a fix to only disable ZCopy when no FMR facility. I don't have time right now to add support for fast memory registeration when FMR is not available - but Steve said he'll help with implementing it. - Amir (I removed this from the linux-kernel list) I probably won't get to adding fast memory registration until July/August. Steve. -- To unsubscribe from this list: send the line unsubscribe linux-rdma in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[infiniband-diags] [1/4] add --diff support to iblinkinfo
Hi Sasha,
Similar to ibnetdiscover, this patch support a --diff option in
iblinkinfo.
Al
--
Albert Chu
ch...@llnl.gov
Computer Scientist
High Performance Systems Division
Lawrence Livermore National Laboratory
---BeginMessage---
Signed-off-by: Albert Chu ch...@llnl.gov
---
infiniband-diags/man/iblinkinfo.8 |8 ++
infiniband-diags/src/iblinkinfo.c | 221 +---
2 files changed, 210 insertions(+), 19 deletions(-)
diff --git a/infiniband-diags/man/iblinkinfo.8
b/infiniband-diags/man/iblinkinfo.8
index f184edf..b91afbd 100644
--- a/infiniband-diags/man/iblinkinfo.8
+++ b/infiniband-diags/man/iblinkinfo.8
@@ -50,6 +50,14 @@ fabrics or a previous state of a fabric. Cannot be used if
user
specifies a directo route path. See
.B ibnetdiscover
for information on caching ibnetdiscover output.
+.TP
+\fB\-\-diff\fR filename
+Load cached ibnetdiscover data and do a diff comparison to the current
+network or another cache. A special diff output for iblinkinfo
+output will be displayed showing differences between the old and current
+fabric links. See
+.B ibnetdiscover
+for information on caching ibnetdiscover output.
.SH AUTHOR
.TP
diff --git a/infiniband-diags/src/iblinkinfo.c
b/infiniband-diags/src/iblinkinfo.c
index 029573f..c6092f9 100644
--- a/infiniband-diags/src/iblinkinfo.c
+++ b/infiniband-diags/src/iblinkinfo.c
@@ -53,9 +53,16 @@
#include ibdiag_common.h
+#define DIFF_FLAG_PORT_CONNECTION 0x01
+#define DIFF_FLAG_PORT_STATE 0x02
+
+#define DIFF_FLAG_DEFAULT (DIFF_FLAG_PORT_CONNECTION | DIFF_FLAG_PORT_STATE)
+
static char *node_name_map_file = NULL;
static nn_map_t *node_name_map = NULL;
static char *load_cache_file = NULL;
+static char *diff_cache_file = NULL;
+static unsigned diffcheck_flags = DIFF_FLAG_DEFAULT;
static uint64_t guid = 0;
static char *guid_str = NULL;
@@ -107,7 +114,7 @@ void get_msg(char *width_msg, char *speed_msg, int
msg_size, ibnd_port_t * port)
buf, 64, max_speed));
}
-void print_port(ibnd_node_t * node, ibnd_port_t * port)
+void print_port(ibnd_node_t * node, ibnd_port_t * port, char *out_prefix)
{
char width[64], speed[64], state[64], physstate[64];
char remote_guid_str[256];
@@ -204,21 +211,35 @@ void print_port(ibnd_node_t * node, ibnd_port_t * port)
if (line_mode) {
char *remap = remap_node_name(node_name_map, node-guid,
node-nodedesc);
- printf(0x%016 PRIx64 \%30s\ , node-guid, remap);
+ printf(%s0x%016 PRIx64 \%30s\ ,
+ out_prefix ? out_prefix : ,
+ node-guid, remap);
free(remap);
} else
- printf( );
+ printf(%s , out_prefix ? out_prefix : );
printf(%6d %4d[%2s] ==%s== %s,
node-smalid, port-portnum, ext_port_str, link_str, remote_str);
}
+void print_switch_header(ibnd_node_t *node, int *out_header_flag, char
*out_prefix)
+{
+ if (!(*out_header_flag) !line_mode) {
+ char *remap =
+ remap_node_name(node_name_map, node-guid,
node-nodedesc);
+ printf(%sSwitch 0x%016 PRIx64 %s:\n,
+ out_prefix ? out_prefix : ,
+ node-guid, remap);
+ (*out_header_flag)++;
+ free(remap);
+ }
+}
+
void print_switch(ibnd_node_t * node, void *user_data)
{
int i = 0;
int head_print = 0;
- char *remap =
- remap_node_name(node_name_map, node-guid, node-nodedesc);
+ char *out_prefix = (char *)user_data;
for (i = 1; i = node-numports; i++) {
ibnd_port_t *port = node-ports[i];
@@ -227,15 +248,153 @@ void print_switch(ibnd_node_t * node, void *user_data)
if (!down_links_only ||
mad_get_field(port-info, 0,
IB_PORT_STATE_F) == IB_LINK_DOWN) {
- if (!head_print !line_mode) {
- printf(Switch 0x%016 PRIx64 %s:\n,
- node-guid, remap);
- head_print = 1;
- }
- print_port(node, port);
+ print_switch_header(node, head_print, out_prefix);
+ print_port(node, port, out_prefix);
+ }
+ }
+}
+
+struct iter_diff_data {
+uint32_t diff_flags;
+ibnd_fabric_t *fabric1;
+ibnd_fabric_t *fabric2;
+char *fabric1_prefix;
+char *fabric2_prefix;
+};
+
+void diff_switch_ports(ibnd_node_t * fabric1_node, ibnd_node_t * fabric2_node,
+ int *head_print, struct iter_diff_data *data)
+{
+ int i = 0;
+
+ for (i = 1; i = fabric1_node-numports; i++) {
+ ibnd_port_t *fabric1_port, *fabric2_port;
+ int
[infiniband-diags] [2/4] support --diffcheck in iblinkinfo
Hi Sasha,
Similar to ibnetdiscover, this patch support a --diffcheck option in
iblinkinfo.
Al
--
Albert Chu
ch...@llnl.gov
Computer Scientist
High Performance Systems Division
Lawrence Livermore National Laboratory
---BeginMessage---
Signed-off-by: Albert Chu ch...@llnl.gov
---
infiniband-diags/man/iblinkinfo.8 |5 +
infiniband-diags/src/iblinkinfo.c | 20
2 files changed, 25 insertions(+), 0 deletions(-)
diff --git a/infiniband-diags/man/iblinkinfo.8
b/infiniband-diags/man/iblinkinfo.8
index b91afbd..431ab0e 100644
--- a/infiniband-diags/man/iblinkinfo.8
+++ b/infiniband-diags/man/iblinkinfo.8
@@ -58,6 +58,11 @@ output will be displayed showing differences between the old
and current
fabric links. See
.B ibnetdiscover
for information on caching ibnetdiscover output.
+.TP
+\fB\-\-diffcheck\fR key(s)
+Specify what diff checks should be done in the \fB\-\-diff\fR option above.
+Comma separate multiple diff check key(s). The available diff checks
+are:\fIport\fR = port connections, \fIstate\fR = port state.
.SH AUTHOR
.TP
diff --git a/infiniband-diags/src/iblinkinfo.c
b/infiniband-diags/src/iblinkinfo.c
index c6092f9..a4a8e49 100644
--- a/infiniband-diags/src/iblinkinfo.c
+++ b/infiniband-diags/src/iblinkinfo.c
@@ -400,6 +400,8 @@ int diff_switch(ibnd_node_t * node, ibnd_fabric_t *
orig_fabric,
static int process_opt(void *context, int ch, char *optarg)
{
struct ibnd_config *cfg = context;
+ char *p;
+
switch (ch) {
case 1:
node_name_map_file = strdup(optarg);
@@ -410,6 +412,22 @@ static int process_opt(void *context, int ch, char *optarg)
case 3:
diff_cache_file = strdup(optarg);
break;
+ case 4:
+ diffcheck_flags = 0;
+ p = strtok(optarg, ,);
+ while (p) {
+ if (!strcasecmp(p, port))
+ diffcheck_flags |= DIFF_FLAG_PORT_CONNECTION;
+ else if (!strcasecmp(p, state))
+ diffcheck_flags |= DIFF_FLAG_PORT_STATE;
+ else {
+ fprintf(stderr, invalid diff check key: %s\n,
+ p);
+ return -1;
+ }
+ p = strtok(NULL, ,);
+ }
+ break;
case 'S':
guid_str = optarg;
guid = (uint64_t) strtoull(guid_str, 0, 0);
@@ -480,6 +498,8 @@ int main(int argc, char **argv)
filename of ibnetdiscover cache to load},
{diff, 3, 1, file,
filename of ibnetdiscover cache to diff},
+ {diffcheck, 4, 1, key(s),
+specify checks to execute for --diff},
{outstanding_smps, 'o', 1, NULL,
specify the number of outstanding SMP's which should be
issued during the scan},
--
1.5.4.5
---End Message---
[infiniband-diags] [3/4] Add lid and node description diff options for --diffcheck in iblinkinfo
Hi Sasha,
This patch supports additional lid and node description diffing options
in iblinkinfo. This is similar to the lid and nodescription --diffcheck
options in ibnetdiscover.
Al
--
Albert Chu
ch...@llnl.gov
Computer Scientist
High Performance Systems Division
Lawrence Livermore National Laboratory
---BeginMessage---
Signed-off-by: Albert Chu ch...@llnl.gov
---
infiniband-diags/man/iblinkinfo.8 |7 +++-
infiniband-diags/src/iblinkinfo.c | 53 -
2 files changed, 51 insertions(+), 9 deletions(-)
diff --git a/infiniband-diags/man/iblinkinfo.8
b/infiniband-diags/man/iblinkinfo.8
index 431ab0e..65ea919 100644
--- a/infiniband-diags/man/iblinkinfo.8
+++ b/infiniband-diags/man/iblinkinfo.8
@@ -55,14 +55,17 @@ for information on caching ibnetdiscover output.
Load cached ibnetdiscover data and do a diff comparison to the current
network or another cache. A special diff output for iblinkinfo
output will be displayed showing differences between the old and current
-fabric links. See
+fabric links. Be default, the following are compared for differences:
+port connections and port state. See
.B ibnetdiscover
for information on caching ibnetdiscover output.
.TP
\fB\-\-diffcheck\fR key(s)
Specify what diff checks should be done in the \fB\-\-diff\fR option above.
Comma separate multiple diff check key(s). The available diff checks
-are:\fIport\fR = port connections, \fIstate\fR = port state.
+are:\fIport\fR = port connections, \fIstate\fR = port state, \fIlid\fR = lids,
+\fInodedesc\fR = node descriptions. If \fIport\fR is specified alongside
\fIlid\fR
+or \fInodedesc\fR, remote port lids and node descriptions will also be
compared.
.SH AUTHOR
.TP
diff --git a/infiniband-diags/src/iblinkinfo.c
b/infiniband-diags/src/iblinkinfo.c
index a4a8e49..19adc20 100644
--- a/infiniband-diags/src/iblinkinfo.c
+++ b/infiniband-diags/src/iblinkinfo.c
@@ -55,6 +55,8 @@
#define DIFF_FLAG_PORT_CONNECTION 0x01
#define DIFF_FLAG_PORT_STATE 0x02
+#define DIFF_FLAG_LID 0x04
+#define DIFF_FLAG_NODE_DESCRIPTION 0x08
#define DIFF_FLAG_DEFAULT (DIFF_FLAG_PORT_CONNECTION | DIFF_FLAG_PORT_STATE)
@@ -224,7 +226,7 @@ void print_port(ibnd_node_t * node, ibnd_port_t * port,
char *out_prefix)
void print_switch_header(ibnd_node_t *node, int *out_header_flag, char
*out_prefix)
{
- if (!(*out_header_flag) !line_mode) {
+ if ((!out_header_flag || !(*out_header_flag)) !line_mode) {
char *remap =
remap_node_name(node_name_map, node-guid,
node-nodedesc);
printf(%sSwitch 0x%016 PRIx64 %s:\n,
@@ -308,9 +310,25 @@ void diff_switch_ports(ibnd_node_t * fabric1_node,
ibnd_node_t * fabric2_node,
output_diff++;
}
+ if (data-diff_flags DIFF_FLAG_PORT_CONNECTION
+data-diff_flags DIFF_FLAG_LID
+fabric1_port fabric2_port
+fabric1_port-remoteport fabric2_port-remoteport
+fabric1_port-remoteport-base_lid !=
fabric2_port-remoteport-base_lid)
+ output_diff++;
+
+ if (data-diff_flags DIFF_FLAG_PORT_CONNECTION
+data-diff_flags DIFF_FLAG_NODE_DESCRIPTION
+fabric1_port fabric2_port
+fabric1_port-remoteport fabric2_port-remoteport
+memcmp(fabric1_port-remoteport-node-nodedesc,
+ fabric2_port-remoteport-node-nodedesc,
+ IB_SMP_DATA_SIZE))
+ output_diff++;
+
if (output_diff fabric1_port) {
print_switch_header(fabric1_node,
- head_print,
+ head_print,
NULL);
print_port(fabric1_node,
fabric1_port,
@@ -319,7 +337,7 @@ void diff_switch_ports(ibnd_node_t * fabric1_node,
ibnd_node_t * fabric2_node,
if (output_diff fabric2_port) {
print_switch_header(fabric1_node,
- head_print,
+ head_print,
NULL);
print_port(fabric2_node,
fabric2_port,
@@ -340,7 +358,22 @@ void diff_switch_iter(ibnd_node_t * fabric1_node, void
*iter_user_data)
if (!fabric2_node)
print_switch(fabric1_node, data-fabric1_prefix);
else if (data-diff_flags
-(DIFF_FLAG_PORT_CONNECTION | DIFF_FLAG_PORT_STATE)) {
+(DIFF_FLAG_PORT_CONNECTION | DIFF_FLAG_PORT_STATE
+ | DIFF_FLAG_LID | DIFF_FLAG_NODE_DESCRIPTION)) {
+
+ if ((data-diff_flags DIFF_FLAG_LID
+
[infiniband-diags] [4/4] support --filterdownports in iblinkinfo
Hi Sasha,
This patch supports a new option called --filterdownports. The option
will remove downports from the output if they were previously listed as
down in a cache.
This option is useful for clusters that have unpopulated switch ports.
Many system administrators look for the word Down in the iblinkinfo
output, however, that ability is limited when so many of the ports are
down all the time b/c of unpopulated ports. This option attempts to
remove that limitation for clusters with unpopulated ports.
Al
--
Albert Chu
ch...@llnl.gov
Computer Scientist
High Performance Systems Division
Lawrence Livermore National Laboratory
---BeginMessage---
Signed-off-by: Albert Chu ch...@llnl.gov
---
infiniband-diags/man/iblinkinfo.8 |9
infiniband-diags/src/iblinkinfo.c | 40 +
2 files changed, 49 insertions(+), 0 deletions(-)
diff --git a/infiniband-diags/man/iblinkinfo.8
b/infiniband-diags/man/iblinkinfo.8
index 65ea919..940d008 100644
--- a/infiniband-diags/man/iblinkinfo.8
+++ b/infiniband-diags/man/iblinkinfo.8
@@ -66,6 +66,15 @@ Comma separate multiple diff check key(s). The available
diff checks
are:\fIport\fR = port connections, \fIstate\fR = port state, \fIlid\fR = lids,
\fInodedesc\fR = node descriptions. If \fIport\fR is specified alongside
\fIlid\fR
or \fInodedesc\fR, remote port lids and node descriptions will also be
compared.
+.TP
+\fB\-\-filterdownports\fR filename
+Filter downports indicated in a ibnetdiscover cache. If a port was previously
+indicated as down in the specified cache, and is still down, do not output it
in the
+resulting output. This option may be particularly useful for environments
+where switches are not fully populated, thus much of the default iblinkinfo
+info is considered unuseful. See
+.B ibnetdiscover
+for information on caching ibnetdiscover output.
.SH AUTHOR
.TP
diff --git a/infiniband-diags/src/iblinkinfo.c
b/infiniband-diags/src/iblinkinfo.c
index 19adc20..4b8616e 100644
--- a/infiniband-diags/src/iblinkinfo.c
+++ b/infiniband-diags/src/iblinkinfo.c
@@ -65,6 +65,8 @@ static nn_map_t *node_name_map = NULL;
static char *load_cache_file = NULL;
static char *diff_cache_file = NULL;
static unsigned diffcheck_flags = DIFF_FLAG_DEFAULT;
+static char *filterdownports_cache_file = NULL;
+static ibnd_fabric_t *filterdownports_fabric = NULL;
static uint64_t guid = 0;
static char *guid_str = NULL;
@@ -116,6 +118,30 @@ void get_msg(char *width_msg, char *speed_msg, int
msg_size, ibnd_port_t * port)
buf, 64, max_speed));
}
+int filterdownport_check(ibnd_node_t * node, ibnd_port_t * port)
+{
+ ibnd_node_t *fsw;
+ ibnd_port_t *fport;
+ int fistate;
+
+ fsw = ibnd_find_node_guid(filterdownports_fabric, node-guid);
+
+ if (!fsw)
+ return 0;
+
+ if (port-portnum fsw-numports)
+ return 0;
+
+ fport = fsw-ports[port-portnum];
+
+ if (!fport)
+ return 0;
+
+ fistate = mad_get_field(fport-info, 0, IB_PORT_STATE_F);
+
+ return (fistate == IB_LINK_DOWN) ? 1 : 0;
+}
+
void print_port(ibnd_node_t * node, ibnd_port_t * port, char *out_prefix)
{
char width[64], speed[64], state[64], physstate[64];
@@ -142,6 +168,11 @@ void print_port(ibnd_node_t * node, ibnd_port_t * port,
char *out_prefix)
width_msg[0] = '\0';
speed_msg[0] = '\0';
+ if (istate == IB_LINK_DOWN
+filterdownports_fabric
+filterdownport_check(node,port))
+ return;
+
/* C14-24.2.1 states that a down port allows for invalid data to be
* returned for all PortInfo components except PortState and
* PortPhysicalState */
@@ -467,6 +498,9 @@ static int process_opt(void *context, int ch, char *optarg)
p = strtok(NULL, ,);
}
break;
+ case 5:
+ filterdownports_cache_file = strdup(optarg);
+ break;
case 'S':
guid_str = optarg;
guid = (uint64_t) strtoull(guid_str, 0, 0);
@@ -539,6 +573,8 @@ int main(int argc, char **argv)
filename of ibnetdiscover cache to diff},
{diffcheck, 4, 1, key(s),
specify checks to execute for --diff},
+ {filterdownports, 5, 1, file,
+filename of ibnetdiscover cache to filter downports},
{outstanding_smps, 'o', 1, NULL,
specify the number of outstanding SMP's which should be
issued during the scan},
@@ -591,6 +627,10 @@ int main(int argc, char **argv)
!(diff_fabric = ibnd_load_fabric(diff_cache_file, 0)))
IBERROR(loading cached fabric for diff failed\n);
+ if (filterdownports_cache_file
+ !(filterdownports_fabric =
ibnd_load_fabric(filterdownports_cache_file, 0)))
+ IBERROR(loading
