Improve flow duration precision

Improve precision of flow duration (time a flow has
been installed in a switch) from seconds to nanoseconds
to improve throughput measurement accuracy when using
flow stats messages. This implementation splits the
existing duration fields into duration_sec and
duration_nsec for both flow expiration and flow stats
messages. Passes included unit tests on Ubuntu 8.04.
This commit is contained in:
David Erickson
2009-11-23 14:33:59 -08:00
committed by Glen Gibb
parent cb0c5d2101
commit bc94ac2e58
9 changed files with 301 additions and 26 deletions
+4 -2
View File
@@ -180,7 +180,7 @@ While the system is running, the datapath may be queried about its current state
The switch responds with one or more \verb|OFPT_STATS_REPLY| messages:
\input{struct/ofp_stats_reply}
The only value defined for \verb|flags| in a reply is whether more replies will follow this one - this has the value \verb|0x0001|. To ease implementation, the switch is allowed to send replies with no additional entries. However, it must always send another reply following a message with the ÒmoreÓ flag set. The transaction ids (xid) of replies must always match the request that prompted them.
The only value defined for \verb|flags| in a reply is whether more replies will follow this one - this has the value \verb|0x0001|. To ease implementation, the switch is allowed to send replies with no additional entries. However, it must always send another reply following a message with the more flag set. The transaction ids (xid) of replies must always match the request that prompted them.
\\\\
In both the request and response, the \verb|type| field specifies the kind of information being passed and determines how the \verb|body| field is interpreted:
@@ -206,6 +206,8 @@ The \verb|body| of the reply consists of an array of the following:
\input{struct/ofp_flow_stats}
The fields consist of those provided in the \verb|flow_mod| that created these, plus the table into which the entry was inserted, the packet count, and the byte count.
\\\\
The \verb|duration_sec| and \verb|duration_nsec| fields indicate the elapsed time the flow has been installed in the switch. The total duration in nanoseconds can be computed as \verb|duration_sec|*10$^{9}$ + \verb|duration_nsec|. Implementations are required to provide millisecond precision; higher precision is encouraged where available.
\paragraph{Aggregate Flow Statistics}
Aggregate information about multiple flows is requested with the \verb|OFPST_AGGREGATE| stats request type:
@@ -275,7 +277,7 @@ The \verb|match| and \verb|priority| fields are the same as those used in the fl
The \verb|reason| field is one of the following:
\input{enum/ofp_flow_removed_reason}
The \verb|duration| field indicates the number of seconds the flow was alive (present in the flow table).
The \verb|duration_sec| and \verb|duration_nsec| fields indicate the elapsed time the flow has been installed in the switch. The total duration in nanoseconds can be computed as \verb|duration_sec|*10$^{9}$ + \verb|duration_nsec|. Implementations are required to provide millisecond precision, higher precision is encouraged where available.
\\\\
The \verb|idle_timeout| field is directly copied from the flow mod that created this entry.
\\\\
+4 -2
View File
@@ -1,4 +1,5 @@
\documentclass[10pt]{article}
\usepackage{amsmath}
\usepackage{listings}
\usepackage{hyperref}
\usepackage{fancyhdr}
@@ -121,7 +122,7 @@ Switch designers are free to implement the internals in any way convenient provi
Counters are maintained per-table, per-flow, and per-port. OpenFlow-compliant counters may be implemented in software and maintained by polling hardware counters with more limited ranges.
\\\\
Table \ref{table:counters} contains the required set of counters. Duration refers to the number of seconds a flow has been active. The Receive Errors field includes all explicitly specified errors, including frame, overrun, and CRC errors, plus any others.
Table \ref{table:counters} contains the required set of counters. Duration refers to the time the flow has been installed in the switch. The Receive Errors field includes all explicitly specified errors, including frame, overrun, and CRC errors, plus any others.
\begin{table}[!hbp]
\centering
\footnotesize
@@ -135,7 +136,8 @@ Table \ref{table:counters} contains the required set of counters. Duration refe
\hline \multicolumn{2}{|c|}{Per Flow} \\
\hline Received Packets & 64 \\
\hline Received Bytes & 64 \\
\hline Duration & 32 \\
\hline Duration (seconds) & 32 \\
\hline Duration (nano seconds) & 32 \\
\hline \multicolumn{2}{|c|}{Per Port} \\
\hline Received Packets & 64 \\
\hline Transmitted Packets & 64 \\
+9 -5
View File
@@ -581,9 +581,11 @@ struct ofp_flow_removed {
uint8_t reason; /* One of OFPRR_*. */
uint8_t pad[1]; /* Align to 32-bits. */
uint32_t duration; /* Time flow was alive in seconds. */
uint32_t duration_sec; /* Time flow was alive in seconds. */
uint32_t duration_nsec; /* Time flow was alive in nanoseconds beyond
duration_sec. */
uint16_t idle_timeout; /* Idle timeout from original flow mod. */
uint8_t pad2[6]; /* Align to 64-bits. */
uint8_t pad2[2]; /* Align to 64-bits. */
uint64_t packet_count;
uint64_t byte_count;
};
@@ -748,17 +750,19 @@ struct ofp_flow_stats {
uint8_t table_id; /* ID of table flow came from. */
uint8_t pad;
struct ofp_match match; /* Description of fields. */
uint32_t duration; /* Time flow has been alive in seconds. */
uint32_t duration_sec; /* Time flow has been alive in seconds. */
uint32_t duration_nsec; /* Time flow has been alive in nanoseconds beyond
duration_sec. */
uint16_t priority; /* Priority of the entry. Only meaningful
when this is not an exact-match entry. */
uint16_t idle_timeout; /* Number of seconds idle before expiration. */
uint16_t hard_timeout; /* Number of seconds before expiration. */
uint16_t pad2; /* Pad to 64 bits. */
uint8_t pad2[6]; /* Align to 64-bits. */
uint64_t packet_count; /* Number of packets in flow. */
uint64_t byte_count; /* Number of bytes in flow. */
struct ofp_action_header actions[0]; /* Actions. */
};
OFP_ASSERT(sizeof(struct ofp_flow_stats) == 72);
OFP_ASSERT(sizeof(struct ofp_flow_stats) == 80);
/* Body for ofp_stats_request of type OFPST_AGGREGATE. */
struct ofp_aggregate_stats_request {
+6 -4
View File
@@ -780,10 +780,11 @@ ofp_print_flow_removed(struct ds *string, const void *oh, size_t len UNUSED,
break;
}
ds_put_format(string,
" pri%"PRIu16" secs%"PRIu32" idle%"PRIu16" pkts%"PRIu64" bytes%"PRIu64"\n",
" pri%"PRIu16" secs%"PRIu32" nsecs%"PRIu32" idle%"PRIu16" pkts%"PRIu64" bytes%"PRIu64"\n",
ofe->match.wildcards ? ntohs(ofe->priority) : (uint16_t)-1,
ntohl(ofe->duration), ntohs(ofe->idle_timeout),
ntohll(ofe->packet_count), ntohll(ofe->byte_count));
ntohl(ofe->duration_sec), ntohl(ofe->duration_nsec),
ntohs(ofe->idle_timeout), ntohll(ofe->packet_count),
ntohll(ofe->byte_count));
}
static void
@@ -984,7 +985,8 @@ ofp_flow_stats_reply(struct ds *string, const void *body_, size_t len,
break;
}
ds_put_format(string, " duration=%"PRIu32"s, ", ntohl(fs->duration));
ds_put_format(string, " duration_sec=%"PRIu32"s, ", ntohl(fs->duration_sec));
ds_put_format(string, "duration_nsec=%"PRIu32"s, ", ntohl(fs->duration_nsec));
ds_put_format(string, "table_id=%"PRIu8", ", fs->table_id);
ds_put_format(string, "priority=%"PRIu16", ",
fs->match.wildcards ? ntohs(fs->priority) : (uint16_t)-1);
@@ -0,0 +1,59 @@
#!/usr/bin/perl -w
# test_flow_expired
use strict;
use OF::Includes;
sub my_test {
my ( $sock, $options_ref ) = @_;
my $in_port = $$options_ref{'port_base'};
my $out_port = $in_port + 1;
my $test_pkt = get_default_black_box_pkt( $in_port, $out_port );
my $max_idle = 0x1; # second before flow expiration
my $wildcards = 0x0; # exact match
my $flags = $enums{'OFPFF_SEND_FLOW_REM'}; # want flow expiry
my $flow_mod_pkt =
create_flow_mod_from_udp( $ofp, $test_pkt, $in_port, $out_port, $max_idle,
$flags, $wildcards );
#print HexDump($pkt);
# Send 'flow_mod' message
print $sock $flow_mod_pkt;
my $pkt_len = 0;
my $pkt_total = 0;
my $read_size = 1512;
my $recvd_mesg;
sysread( $sock, $recvd_mesg, $read_size )
|| die "Failed to receive ofp_flow_removed message: $!";
# Inspect message
my $msg_size = length($recvd_mesg);
my $expected_size = $ofp->sizeof('ofp_flow_removed');
compare( "ofp_flow_removed msg size",
length($recvd_mesg), '==', $expected_size );
my $msg = $ofp->unpack( 'ofp_flow_removed', $recvd_mesg );
print Dumper($msg);
compare( "ofp_flow_removed packet_count",
$$msg{'packet_count'}, '==', $pkt_total );
if ( $$msg{'duration_sec'} != 1 ) {
die "Error, duration_sec out of acceptable range";
}
if ( $$msg{'duration_nsec'} == 0 ) {
die "Error, duration_nsec out of acceptable range";
}
}
run_black_box_test( \&my_test, \@ARGV );
@@ -0,0 +1,188 @@
#!/usr/bin/perl -w
# test_flow_stats
use strict;
use OF::Includes;
sub my_test {
my ( $sock, $options_ref ) = @_;
my $port_base = $$options_ref{'port_base'};
my $hdr_args = {
version => get_of_ver(),
type => $enums{'OFPT_STATS_REQUEST'},
length => $ofp->sizeof('ofp_stats_request') +
$ofp->sizeof('ofp_flow_stats_request')
, # should generate automatically!
xid => 0x00000000
};
my $match_args = {
wildcards => 0x3ef,
in_port => 0,
dl_src => [],
dl_dst => [],
dl_vlan => 0,
dl_type => 0x800,
nw_src => ( NF2::IP_hdr::getIP("192.168.200.1") )[0],
nw_dst => ( NF2::IP_hdr::getIP("192.168.201.2") )[0],
nw_proto => 0,
tp_src => 0,
tp_dst => 0
};
my $stats_request_args = {
header => $hdr_args,
type => $enums{'OFPST_FLOW'},
flags => 0
};
my $body_args = {
match => $match_args,
table_id => 0xff, #match all tables
out_port => $enums{'OFPP_NONE'},
};
my $body = $ofp->pack( 'ofp_flow_stats_request', $body_args );
my $stats_request = $ofp->pack( 'ofp_stats_request', $stats_request_args );
my $mesg = $stats_request . $body;
# Send 'stats_request' message
print $sock $mesg;
# Should add timeout here - will crash if no reply
my $recvd_mesg;
sysread( $sock, $recvd_mesg, 1512 ) || die "Failed to receive message: $!";
# Inspect message
my $msg_size = length($recvd_mesg);
my $msg = $ofp->unpack( 'ofp_stats_reply', $recvd_mesg );
#print HexDump ($recvd_mesg);
print Dumper($msg);
# Verify fields
verify_header( $msg, 'OFPT_STATS_REPLY', $msg_size );
#----------------------
# add flow mod, send pkt
{
my $in_port_offset = 0;
my $out_port_offset = 1;
my $wildcards = 0x3ef;
my $wait = 5;
forward_simple( $ofp, $sock, $options_ref, $in_port_offset,
$out_port_offset, $wildcards, 'any', 1 );
}
sleep .1;
# Send 'stats_request' message
print $sock $mesg;
# Should add timeout here - will crash if no reply
my $recvd_mesg;
sysread( $sock, $recvd_mesg, 1512 ) || die "Failed to receive message: $!";
# Inspect message
my $msg_size = length($recvd_mesg);
my $msg = $ofp->unpack( 'ofp_stats_reply', $recvd_mesg );
# Strip off the header from the received message
$recvd_mesg = substr( $recvd_mesg, $ofp->sizeof('ofp_stats_reply') );
# Unpack each of the ofp_flow_stats messages
my $flow_stats_len = $ofp->sizeof('ofp_flow_stats');
while ( length($recvd_mesg) > 0 ) {
if ( length($recvd_mesg) < $flow_stats_len ) {
\die "Error: Partial flow stats message received";
}
my $flow_stats = $ofp->unpack( 'ofp_flow_stats', $recvd_mesg );
push @{ $msg->{'body'} }, $flow_stats;
$recvd_mesg = substr( $recvd_mesg, $flow_stats->{'length'} );
}
#print HexDump ($recvd_mesg);
print Dumper($msg);
# Verify fields
verify_header( $msg, 'OFPT_STATS_REPLY', $msg_size );
# Ensure that we got back one ofp_flow_stats body
compare(
"stats_reply flow_stats count",
scalar( @{ $msg->{'body'} } ),
'==', 1
);
if ( $msg->{'body'}[0]->{'duration_sec'} != 0 ) {
die "Error, duration_sec out of acceptable range";
}
if ( $msg->{'body'}[0]->{'duration_nsec'} < 100000000
|| $msg->{'body'}[0]->{'duration_nsec'} > 600000000 )
{
die "Error, duration_nsec out of acceptable range";
}
# Sleep 1 more second to test duration_sec
sleep 1.0;
# Send 'stats_request' message
print $sock $mesg;
# Should add timeout here - will crash if no reply
my $recvd_mesg;
sysread( $sock, $recvd_mesg, 1512 ) || die "Failed to receive message: $!";
# Inspect message
my $msg_size = length($recvd_mesg);
my $msg = $ofp->unpack( 'ofp_stats_reply', $recvd_mesg );
# Strip off the header from the received message
$recvd_mesg = substr( $recvd_mesg, $ofp->sizeof('ofp_stats_reply') );
# Unpack each of the ofp_flow_stats messages
my $flow_stats_len = $ofp->sizeof('ofp_flow_stats');
while ( length($recvd_mesg) > 0 ) {
if ( length($recvd_mesg) < $flow_stats_len ) {
\die "Error: Partial flow stats message received";
}
my $flow_stats = $ofp->unpack( 'ofp_flow_stats', $recvd_mesg );
push @{ $msg->{'body'} }, $flow_stats;
$recvd_mesg = substr( $recvd_mesg, $flow_stats->{'length'} );
}
#print HexDump ($recvd_mesg);
print Dumper($msg);
# Verify fields
verify_header( $msg, 'OFPT_STATS_REPLY', $msg_size );
# Ensure that we got back one ofp_flow_stats body
compare(
"stats_reply flow_stats count",
scalar( @{ $msg->{'body'} } ),
'==', 1
);
if ( $msg->{'body'}[0]->{'duration_sec'} != 1 ) {
die "Error, duration_sec out of acceptable range";
}
if ( $msg->{'body'}[0]->{'duration_nsec'} < 100000000
|| $msg->{'body'}[0]->{'duration_nsec'} > 600000000 )
{
die "Error, duration_nsec out of acceptable range";
}
}
run_black_box_test( \&my_test, \@ARGV );
@@ -8,12 +8,14 @@ test_packet_out/run.pl
test_switch_config/run.pl
test_flow_expired/run.pl
test_flow_expired_idle_timeout/run.pl
test_flow_expired_precision/run.pl
test_flow_expired_send_flow_exp/run.pl
test_miss_send_length/run.pl
# Read State Tests
test_port_stats/run.pl
test_flow_stats/run.pl
test_flow_stats_precision/run.pl
## Forwarding Tests
test_forward_any_port/run.pl
+8 -2
View File
@@ -732,6 +732,8 @@ dp_send_flow_end(struct datapath *dp, struct sw_flow *flow,
{
struct ofpbuf *buffer;
struct ofp_flow_removed *ofr;
uint64_t tdiff = time_msec() - flow->created;
uint32_t sec = tdiff / 1000;
if (!flow->send_flow_rem) {
return;
@@ -751,7 +753,8 @@ dp_send_flow_end(struct datapath *dp, struct sw_flow *flow,
ofr->priority = htons(flow->priority);
ofr->reason = reason;
ofr->duration = htonl((time_msec()-flow->created)/1000);
ofr->duration_sec = htonl(sec);
ofr->duration_nsec = htonl((tdiff -(sec*1000))*1000000);
ofr->idle_timeout = htons(flow->idle_timeout);
ofr->packet_count = htonll(flow->packet_count);
@@ -779,6 +782,8 @@ fill_flow_stats(struct ofpbuf *buffer, struct sw_flow *flow,
{
struct ofp_flow_stats *ofs;
int length = sizeof *ofs + flow->sf_acts->actions_len;
uint64_t tdiff = now - flow->created;
uint32_t sec = tdiff / 1000;
ofs = ofpbuf_put_uninit(buffer, length);
ofs->length = htons(length);
ofs->table_id = table_idx;
@@ -795,7 +800,8 @@ fill_flow_stats(struct ofpbuf *buffer, struct sw_flow *flow,
ofs->match.dl_vlan_pcp = flow->key.flow.dl_vlan_pcp;
ofs->match.tp_src = flow->key.flow.tp_src;
ofs->match.tp_dst = flow->key.flow.tp_dst;
ofs->duration = htonl((now - flow->created) / 1000);
ofs->duration_sec = htonl(sec);
ofs->duration_nsec = htonl((tdiff - (sec*1000))*1000000);
ofs->priority = htons(flow->priority);
ofs->idle_timeout = htons(flow->idle_timeout);
ofs->hard_timeout = htons(flow->hard_timeout);
@@ -502,7 +502,8 @@ static gint ofp_flow_stats_reply = -1;
static gint ofp_flow_stats_reply_length = -1;
static gint ofp_flow_stats_reply_table_id = -1;
/* field: ofp_match */
static gint ofp_flow_stats_reply_duration = -1;
static gint ofp_flow_stats_reply_duration_sec = -1;
static gint ofp_flow_stats_reply_duration_nsec = -1;
static gint ofp_flow_stats_reply_priority = -1;
static gint ofp_flow_stats_reply_idle_timeout = -1;
static gint ofp_flow_stats_reply_hard_timeout = -1;
@@ -567,7 +568,8 @@ static gint ofp_flow_removed = -1;
/* field: ofp_match */
static gint ofp_flow_removed_priority = -1;
static gint ofp_flow_removed_reason = -1;
static gint ofp_flow_removed_duration = -1;
static gint ofp_flow_removed_duration_sec = -1;
static gint ofp_flow_removed_duration_nsec = -1;
static gint ofp_flow_removed_idle_timeout = -1;
static gint ofp_flow_removed_packet_count = -1;
static gint ofp_flow_removed_byte_count = -1;
@@ -1311,10 +1313,13 @@ void proto_register_openflow()
{ &ofp_flow_removed_reason,
{ "Reason", "of.fe_reason", FT_UINT8, BASE_DEC, VALS(names_ofp_flow_removed_reason), NO_MASK, "Reason", HFILL } },
{ &ofp_flow_removed_duration,
{ "Flow Duration (sec)", "of.fe_duration", FT_UINT32, BASE_DEC, NO_STRINGS, NO_MASK, "Time Flow was Alive (sec)", HFILL } },
{ &ofp_flow_removed_duration_sec,
{ "Flow Duration (sec)", "of.fe_duration_sec", FT_UINT32, BASE_DEC, NO_STRINGS, NO_MASK, "Time Flow was Alive (sec)", HFILL } },
{ &ofp_flow_removed_idle_timeout,
{ &ofp_flow_removed_duration_nsec,
{ "Flow Duration (nsec)", "of.fe_duration_nsec", FT_UINT32, BASE_DEC, NO_STRINGS, NO_MASK, "Time Flow was Alive (nsec)", HFILL } },
{ &ofp_flow_removed_idle_timeout,
{ "Idle Time (sec) Before Discarding", "of.fe_idle_timeout", FT_UINT16, BASE_DEC, NO_STRINGS, NO_MASK, "Idle Time (sec) Before Discarding", HFILL } },
{ &ofp_flow_removed_packet_count,
@@ -1499,8 +1504,11 @@ void proto_register_openflow()
{ &ofp_flow_stats_reply_table_id,
{ "Table ID", "of.stats_flow_table_id", FT_UINT8, BASE_DEC, NO_STRINGS, NO_MASK, "Table ID", HFILL } },
{ &ofp_flow_stats_reply_duration,
{ "Flow Duration (sec)", "of.stats_flow_duration", FT_UINT32, BASE_DEC, NO_STRINGS, NO_MASK, "Time Flow has Been Alive (sec)", HFILL } },
{ &ofp_flow_stats_reply_duration_sec,
{ "Flow Duration (sec)", "of.stats_flow_duration_sec", FT_UINT32, BASE_DEC, NO_STRINGS, NO_MASK, "Time Flow has Been Alive (sec)", HFILL } },
{ &ofp_flow_stats_reply_duration_nsec,
{ "Flow Duration (nsec)", "of.stats_flow_duration_nsec", FT_UINT32, BASE_DEC, NO_STRINGS, NO_MASK, "Time Flow has Been Alive (nsec)", HFILL } },
{ &ofp_flow_stats_reply_priority,
{ "Priority", "of.stats_flow_priority", FT_UINT16, BASE_DEC, NO_STRINGS, NO_MASK, "Priority", HFILL } },
@@ -2637,9 +2645,10 @@ static void dissect_openflow_message(tvbuff_t *tvb, packet_info *pinfo, proto_tr
add_child(type_tree, ofp_flow_removed_priority, tvb, &offset, 2);
add_child(type_tree, ofp_flow_removed_reason, tvb, &offset, 1);
dissect_pad(type_tree, &offset, 1);
add_child(type_tree, ofp_flow_removed_duration, tvb, &offset, 4);
add_child(type_tree, ofp_flow_removed_duration_sec, tvb, &offset, 4);
add_child(type_tree, ofp_flow_removed_duration_nsec, tvb, &offset, 4);
add_child(type_tree, ofp_flow_removed_idle_timeout, tvb, &offset, 2);
dissect_pad(type_tree, &offset, 6);
dissect_pad(type_tree, &offset, 2);
add_child(type_tree, ofp_flow_removed_packet_count, tvb, &offset, 8);
add_child(type_tree, ofp_flow_removed_byte_count, tvb, &offset, 8);
break;
@@ -2833,11 +2842,12 @@ static void dissect_openflow_message(tvbuff_t *tvb, packet_info *pinfo, proto_tr
add_child(flow_tree, ofp_flow_stats_reply_table_id, tvb, &offset, 1);
dissect_pad(flow_tree, &offset, 1);
dissect_match(flow_tree, flow_item, tvb, pinfo, &offset);
add_child(flow_tree, ofp_flow_stats_reply_duration, tvb, &offset, 4);
add_child(flow_tree, ofp_flow_stats_reply_duration_sec, tvb, &offset, 4);
add_child(flow_tree, ofp_flow_stats_reply_duration_nsec, tvb, &offset, 4);
add_child(flow_tree, ofp_flow_stats_reply_priority, tvb, &offset, 2);
add_child(flow_tree, ofp_flow_stats_reply_idle_timeout, tvb, &offset, 2);
add_child(flow_tree, ofp_flow_stats_reply_hard_timeout, tvb, &offset, 2);
dissect_pad(flow_tree, &offset, 2);
dissect_pad(flow_tree, &offset, 6);
add_child(flow_tree, ofp_flow_stats_reply_packet_count, tvb, &offset, 8);
add_child(flow_tree, ofp_flow_stats_reply_byte_count, tvb, &offset, 8);