Olivier Lemasle created CLOUDSTACK-5404: -------------------------------------------
Summary: Network usages (bytes sent/received) are saved in the wrong timezone Key: CLOUDSTACK-5404 URL: https://issues.apache.org/jira/browse/CLOUDSTACK-5404 Project: CloudStack Issue Type: Bug Security Level: Public (Anyone can view this level - this is the default.) Components: Usage Affects Versions: 4.2.0 Reporter: Olivier Lemasle On ACS 4.2, usages with types 4 and 5 (Network bytes sent / Network bytes received) are not stored in database (table cloud_usage.cloud_usage) with the same timezone than the other usages. On my CloudStack 4.2 installation (using GMT+1), network usages appear in database one hour later than the other usage types: {noformat} mysql> select usage_type, max(end_date) from cloud_usage group by usage_type; +------------+---------------------+ | usage_type | max(end_date) | +------------+---------------------+ | 1 | 2013-12-06 14:59:59 | | 2 | 2013-12-06 14:59:59 | | 3 | 2013-12-06 14:59:59 | | 4 | 2013-12-06 15:59:59 | | 5 | 2013-12-06 15:59:59 | | 6 | 2013-12-06 14:59:59 | | 13 | 2013-12-06 14:59:59 | +------------+---------------------+ 7 rows in set (0.00 sec) {noformat} In fact: - for network bytes sent/received, usages are stored in local timezone (in my case GMT+1) - for every other usage types, usages are stored in GMT. I checked the SQL requests and here are two consecutive requests: {code:sql} INSERT INTO cloud_usage.cloud_usage (zone_id, account_id, domain_id, description, usage_display, usage_type, raw_usage, vm_instance_id, vm_name, offering_id, template_id, usage_id, type, size, network_id, start_date, end_date, virtual_size) VALUES (1,2,1,'network bytes received for Host: 4','0 bytes received',5,0.0,null,null, null, null, 4,'DomainRouter',null,204,'2013-12-06 15:00:00','2013-12-06 15:59:59',null) INSERT INTO cloud_usage (cloud_usage.zone_id, cloud_usage.account_id, cloud_usage.domain_id, cloud_usage.description, cloud_usage.usage_display, cloud_usage.usage_type, cloud_usage.raw_usage, cloud_usage.vm_instance_id, cloud_usage.vm_name, cloud_usage.offering_id, cloud_usage.template_id, cloud_usage.usage_id, cloud_usage.type, cloud_usage.size, cloud_usage.virtual_size, cloud_usage.network_id, cloud_usage.start_date, cloud_usage.end_date) VALUES (1, 2, 1, _binary'Volume Id: 3 usage time', _binary'1 Hrs', 6, 1.0, null, null, null, null, 3, null, 21474836480, null, null, '2013-12-06 14:00:00', '2013-12-06 14:59:59') {code} The first, for Network bytes sent, is a request made in com.cloud.usage.dao.UsageDaoImpl.saveUsageRecords(List<UsageVO>), and the second is done with com.cloud.utils.db.GenericDaoBase.persist(T). The issue comes from the fix for CLOUDSTACK-2707, where, for performance reasons, multiple calls to persist() in com.cloud.usage.parser.NetworkUsageParser were replaced by a single call to saveUsageRecords() to process the insertions as a batch. So, NetworkUsageParser uses this saveUsageRecords() function with no special timezone management: {code:java} pstmt.setTimestamp(16, new Timestamp(usageRecord.getStartDate().getTime())); pstmt.setTimestamp(17, new Timestamp(usageRecord.getEndDate().getTime())); {code} whereas for other usages, parsers (VMInstanceUsageParser, VolumeUsageParser, NetworkOfferingUsageParser,...) use persist() which has a special timezone management converting every date in GMT: {code:java} else if (attr.field.getType() == Date.class) { final Date date = (Date)value; if (date == null) { pstmt.setObject(j, null); return; } if (attr.is(Attribute.Flag.Date)) { pstmt.setString(j, DateUtil.getDateDisplayString(s_gmtTimeZone, date)); } else if (attr.is(Attribute.Flag.TimeStamp)) { pstmt.setString(j, DateUtil.getDateDisplayString(s_gmtTimeZone, date)); } else if (attr.is(Attribute.Flag.Time)) { pstmt.setString(j, DateUtil.getDateDisplayString(s_gmtTimeZone, date)); } } {code} -- This message was sent by Atlassian JIRA (v6.1#6144)