SunnyQjm/mahimahi
1
0 Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity

Merge pull request #1 from MacroLau/dev/init

Browse Source 
add init mahimahi code
This commit is contained in:
MacroLau authored and GitHub committed 2022-02-25 09:51:40 +08:00
2 parents 19a023576d 1f45cb0f9e
commit a0d0cf9ed8
180 files changed
+4394332 -1
Download Patch File Download Diff File Expand all files Collapse all files

No files matched your search

COPYING
+674
View File
@@ -0,0 +1,674 @@
GNU GENERAL PUBLIC LICENSE
Version 3, 29 June 2007
Copyright (C) 2007 Free Software Foundation, Inc. <http://fsf.org/>
Everyone is permitted to copy and distribute verbatim copies
of this license document, but changing it is not allowed.
Preamble
The GNU General Public License is a free, copyleft license for
software and other kinds of works.
The licenses for most software and other practical works are designed
to take away your freedom to share and change the works. By contrast,
the GNU General Public License is intended to guarantee your freedom to
share and change all versions of a program--to make sure it remains free
software for all its users. We, the Free Software Foundation, use the
GNU General Public License for most of our software; it applies also to
any other work released this way by its authors. You can apply it to
your programs, too.
When we speak of free software, we are referring to freedom, not
price. Our General Public Licenses are designed to make sure that you
have the freedom to distribute copies of free software (and charge for
them if you wish), that you receive source code or can get it if you
want it, that you can change the software or use pieces of it in new
free programs, and that you know you can do these things.
To protect your rights, we need to prevent others from denying you
these rights or asking you to surrender the rights. Therefore, you have
certain responsibilities if you distribute copies of the software, or if
you modify it: responsibilities to respect the freedom of others.
For example, if you distribute copies of such a program, whether
gratis or for a fee, you must pass on to the recipients the same
freedoms that you received. You must make sure that they, too, receive
or can get the source code. And you must show them these terms so they
know their rights.
Developers that use the GNU GPL protect your rights with two steps:
(1) assert copyright on the software, and (2) offer you this License
giving you legal permission to copy, distribute and/or modify it.
For the developers' and authors' protection, the GPL clearly explains
that there is no warranty for this free software. For both users' and
authors' sake, the GPL requires that modified versions be marked as
changed, so that their problems will not be attributed erroneously to
authors of previous versions.
Some devices are designed to deny users access to install or run
modified versions of the software inside them, although the manufacturer
can do so. This is fundamentally incompatible with the aim of
protecting users' freedom to change the software. The systematic
pattern of such abuse occurs in the area of products for individuals to
use, which is precisely where it is most unacceptable. Therefore, we
have designed this version of the GPL to prohibit the practice for those
products. If such problems arise substantially in other domains, we
stand ready to extend this provision to those domains in future versions
of the GPL, as needed to protect the freedom of users.
Finally, every program is threatened constantly by software patents.
States should not allow patents to restrict development and use of
software on general-purpose computers, but in those that do, we wish to
avoid the special danger that patents applied to a free program could
make it effectively proprietary. To prevent this, the GPL assures that
patents cannot be used to render the program non-free.
The precise terms and conditions for copying, distribution and
modification follow.
TERMS AND CONDITIONS
0. Definitions.
"This License" refers to version 3 of the GNU General Public License.
"Copyright" also means copyright-like laws that apply to other kinds of
works, such as semiconductor masks.
"The Program" refers to any copyrightable work licensed under this
License. Each licensee is addressed as "you". "Licensees" and
"recipients" may be individuals or organizations.
To "modify" a work means to copy from or adapt all or part of the work
in a fashion requiring copyright permission, other than the making of an
exact copy. The resulting work is called a "modified version" of the
earlier work or a work "based on" the earlier work.
A "covered work" means either the unmodified Program or a work based
on the Program.
To "propagate" a work means to do anything with it that, without
permission, would make you directly or secondarily liable for
infringement under applicable copyright law, except executing it on a
computer or modifying a private copy. Propagation includes copying,
distribution (with or without modification), making available to the
public, and in some countries other activities as well.
To "convey" a work means any kind of propagation that enables other
parties to make or receive copies. Mere interaction with a user through
a computer network, with no transfer of a copy, is not conveying.
An interactive user interface displays "Appropriate Legal Notices"
to the extent that it includes a convenient and prominently visible
feature that (1) displays an appropriate copyright notice, and (2)
tells the user that there is no warranty for the work (except to the
extent that warranties are provided), that licensees may convey the
work under this License, and how to view a copy of this License. If
the interface presents a list of user commands or options, such as a
menu, a prominent item in the list meets this criterion.
1. Source Code.
The "source code" for a work means the preferred form of the work
for making modifications to it. "Object code" means any non-source
form of a work.
A "Standard Interface" means an interface that either is an official
standard defined by a recognized standards body, or, in the case of
interfaces specified for a particular programming language, one that
is widely used among developers working in that language.
The "System Libraries" of an executable work include anything, other
than the work as a whole, that (a) is included in the normal form of
packaging a Major Component, but which is not part of that Major
Component, and (b) serves only to enable use of the work with that
Major Component, or to implement a Standard Interface for which an
implementation is available to the public in source code form. A
"Major Component", in this context, means a major essential component
(kernel, window system, and so on) of the specific operating system
(if any) on which the executable work runs, or a compiler used to
produce the work, or an object code interpreter used to run it.
The "Corresponding Source" for a work in object code form means all
the source code needed to generate, install, and (for an executable
work) run the object code and to modify the work, including scripts to
control those activities. However, it does not include the work's
System Libraries, or general-purpose tools or generally available free
programs which are used unmodified in performing those activities but
which are not part of the work. For example, Corresponding Source
includes interface definition files associated with source files for
the work, and the source code for shared libraries and dynamically
linked subprograms that the work is specifically designed to require,
such as by intimate data communication or control flow between those
subprograms and other parts of the work.
The Corresponding Source need not include anything that users
can regenerate automatically from other parts of the Corresponding
Source.
The Corresponding Source for a work in source code form is that
same work.
2. Basic Permissions.
All rights granted under this License are granted for the term of
copyright on the Program, and are irrevocable provided the stated
conditions are met. This License explicitly affirms your unlimited
permission to run the unmodified Program. The output from running a
covered work is covered by this License only if the output, given its
content, constitutes a covered work. This License acknowledges your
rights of fair use or other equivalent, as provided by copyright law.
You may make, run and propagate covered works that you do not
convey, without conditions so long as your license otherwise remains
in force. You may convey covered works to others for the sole purpose
of having them make modifications exclusively for you, or provide you
with facilities for running those works, provided that you comply with
the terms of this License in conveying all material for which you do
not control copyright. Those thus making or running the covered works
for you must do so exclusively on your behalf, under your direction
and control, on terms that prohibit them from making any copies of
your copyrighted material outside their relationship with you.
Conveying under any other circumstances is permitted solely under
the conditions stated below. Sublicensing is not allowed; section 10
makes it unnecessary.
3. Protecting Users' Legal Rights From Anti-Circumvention Law.
No covered work shall be deemed part of an effective technological
measure under any applicable law fulfilling obligations under article
11 of the WIPO copyright treaty adopted on 20 December 1996, or
similar laws prohibiting or restricting circumvention of such
measures.
When you convey a covered work, you waive any legal power to forbid
circumvention of technological measures to the extent such circumvention
is effected by exercising rights under this License with respect to
the covered work, and you disclaim any intention to limit operation or
modification of the work as a means of enforcing, against the work's
users, your or third parties' legal rights to forbid circumvention of
technological measures.
4. Conveying Verbatim Copies.
You may convey verbatim copies of the Program's source code as you
receive it, in any medium, provided that you conspicuously and
appropriately publish on each copy an appropriate copyright notice;
keep intact all notices stating that this License and any
non-permissive terms added in accord with section 7 apply to the code;
keep intact all notices of the absence of any warranty; and give all
recipients a copy of this License along with the Program.
You may charge any price or no price for each copy that you convey,
and you may offer support or warranty protection for a fee.
5. Conveying Modified Source Versions.
You may convey a work based on the Program, or the modifications to
produce it from the Program, in the form of source code under the
terms of section 4, provided that you also meet all of these conditions:
a) The work must carry prominent notices stating that you modified
it, and giving a relevant date.
b) The work must carry prominent notices stating that it is
released under this License and any conditions added under section
7. This requirement modifies the requirement in section 4 to
"keep intact all notices".
c) You must license the entire work, as a whole, under this
License to anyone who comes into possession of a copy. This
License will therefore apply, along with any applicable section 7
additional terms, to the whole of the work, and all its parts,
regardless of how they are packaged. This License gives no
permission to license the work in any other way, but it does not
invalidate such permission if you have separately received it.
d) If the work has interactive user interfaces, each must display
Appropriate Legal Notices; however, if the Program has interactive
interfaces that do not display Appropriate Legal Notices, your
work need not make them do so.
A compilation of a covered work with other separate and independent
works, which are not by their nature extensions of the covered work,
and which are not combined with it such as to form a larger program,
in or on a volume of a storage or distribution medium, is called an
"aggregate" if the compilation and its resulting copyright are not
used to limit the access or legal rights of the compilation's users
beyond what the individual works permit. Inclusion of a covered work
in an aggregate does not cause this License to apply to the other
parts of the aggregate.
6. Conveying Non-Source Forms.
You may convey a covered work in object code form under the terms
of sections 4 and 5, provided that you also convey the
machine-readable Corresponding Source under the terms of this License,
in one of these ways:
a) Convey the object code in, or embodied in, a physical product
(including a physical distribution medium), accompanied by the
Corresponding Source fixed on a durable physical medium
customarily used for software interchange.
b) Convey the object code in, or embodied in, a physical product
(including a physical distribution medium), accompanied by a
written offer, valid for at least three years and valid for as
long as you offer spare parts or customer support for that product
model, to give anyone who possesses the object code either (1) a
copy of the Corresponding Source for all the software in the
product that is covered by this License, on a durable physical
medium customarily used for software interchange, for a price no
more than your reasonable cost of physically performing this
conveying of source, or (2) access to copy the
Corresponding Source from a network server at no charge.
c) Convey individual copies of the object code with a copy of the
written offer to provide the Corresponding Source. This
alternative is allowed only occasionally and noncommercially, and
only if you received the object code with such an offer, in accord
with subsection 6b.
d) Convey the object code by offering access from a designated
place (gratis or for a charge), and offer equivalent access to the
Corresponding Source in the same way through the same place at no
further charge. You need not require recipients to copy the
Corresponding Source along with the object code. If the place to
copy the object code is a network server, the Corresponding Source
may be on a different server (operated by you or a third party)
that supports equivalent copying facilities, provided you maintain
clear directions next to the object code saying where to find the
Corresponding Source. Regardless of what server hosts the
Corresponding Source, you remain obligated to ensure that it is
available for as long as needed to satisfy these requirements.
e) Convey the object code using peer-to-peer transmission, provided
you inform other peers where the object code and Corresponding
Source of the work are being offered to the general public at no
charge under subsection 6d.
A separable portion of the object code, whose source code is excluded
from the Corresponding Source as a System Library, need not be
included in conveying the object code work.
A "User Product" is either (1) a "consumer product", which means any
tangible personal property which is normally used for personal, family,
or household purposes, or (2) anything designed or sold for incorporation
into a dwelling. In determining whether a product is a consumer product,
doubtful cases shall be resolved in favor of coverage. For a particular
product received by a particular user, "normally used" refers to a
typical or common use of that class of product, regardless of the status
of the particular user or of the way in which the particular user
actually uses, or expects or is expected to use, the product. A product
is a consumer product regardless of whether the product has substantial
commercial, industrial or non-consumer uses, unless such uses represent
the only significant mode of use of the product.
"Installation Information" for a User Product means any methods,
procedures, authorization keys, or other information required to install
and execute modified versions of a covered work in that User Product from
a modified version of its Corresponding Source. The information must
suffice to ensure that the continued functioning of the modified object
code is in no case prevented or interfered with solely because
modification has been made.
If you convey an object code work under this section in, or with, or
specifically for use in, a User Product, and the conveying occurs as
part of a transaction in which the right of possession and use of the
User Product is transferred to the recipient in perpetuity or for a
fixed term (regardless of how the transaction is characterized), the
Corresponding Source conveyed under this section must be accompanied
by the Installation Information. But this requirement does not apply
if neither you nor any third party retains the ability to install
modified object code on the User Product (for example, the work has
been installed in ROM).
The requirement to provide Installation Information does not include a
requirement to continue to provide support service, warranty, or updates
for a work that has been modified or installed by the recipient, or for
the User Product in which it has been modified or installed. Access to a
network may be denied when the modification itself materially and
adversely affects the operation of the network or violates the rules and
protocols for communication across the network.
Corresponding Source conveyed, and Installation Information provided,
in accord with this section must be in a format that is publicly
documented (and with an implementation available to the public in
source code form), and must require no special password or key for
unpacking, reading or copying.
7. Additional Terms.
"Additional permissions" are terms that supplement the terms of this
License by making exceptions from one or more of its conditions.
Additional permissions that are applicable to the entire Program shall
be treated as though they were included in this License, to the extent
that they are valid under applicable law. If additional permissions
apply only to part of the Program, that part may be used separately
under those permissions, but the entire Program remains governed by
this License without regard to the additional permissions.
When you convey a copy of a covered work, you may at your option
remove any additional permissions from that copy, or from any part of
it. (Additional permissions may be written to require their own
removal in certain cases when you modify the work.) You may place
additional permissions on material, added by you to a covered work,
for which you have or can give appropriate copyright permission.
Notwithstanding any other provision of this License, for material you
add to a covered work, you may (if authorized by the copyright holders of
that material) supplement the terms of this License with terms:
a) Disclaiming warranty or limiting liability differently from the
terms of sections 15 and 16 of this License; or
b) Requiring preservation of specified reasonable legal notices or
author attributions in that material or in the Appropriate Legal
Notices displayed by works containing it; or
c) Prohibiting misrepresentation of the origin of that material, or
requiring that modified versions of such material be marked in
reasonable ways as different from the original version; or
d) Limiting the use for publicity purposes of names of licensors or
authors of the material; or
e) Declining to grant rights under trademark law for use of some
trade names, trademarks, or service marks; or
f) Requiring indemnification of licensors and authors of that
material by anyone who conveys the material (or modified versions of
it) with contractual assumptions of liability to the recipient, for
any liability that these contractual assumptions directly impose on
those licensors and authors.
All other non-permissive additional terms are considered "further
restrictions" within the meaning of section 10. If the Program as you
received it, or any part of it, contains a notice stating that it is
governed by this License along with a term that is a further
restriction, you may remove that term. If a license document contains
a further restriction but permits relicensing or conveying under this
License, you may add to a covered work material governed by the terms
of that license document, provided that the further restriction does
not survive such relicensing or conveying.
If you add terms to a covered work in accord with this section, you
must place, in the relevant source files, a statement of the
additional terms that apply to those files, or a notice indicating
where to find the applicable terms.
Additional terms, permissive or non-permissive, may be stated in the
form of a separately written license, or stated as exceptions;
the above requirements apply either way.
8. Termination.
You may not propagate or modify a covered work except as expressly
provided under this License. Any attempt otherwise to propagate or
modify it is void, and will automatically terminate your rights under
this License (including any patent licenses granted under the third
paragraph of section 11).
However, if you cease all violation of this License, then your
license from a particular copyright holder is reinstated (a)
provisionally, unless and until the copyright holder explicitly and
finally terminates your license, and (b) permanently, if the copyright
holder fails to notify you of the violation by some reasonable means
prior to 60 days after the cessation.
Moreover, your license from a particular copyright holder is
reinstated permanently if the copyright holder notifies you of the
violation by some reasonable means, this is the first time you have
received notice of violation of this License (for any work) from that
copyright holder, and you cure the violation prior to 30 days after
your receipt of the notice.
Termination of your rights under this section does not terminate the
licenses of parties who have received copies or rights from you under
this License. If your rights have been terminated and not permanently
reinstated, you do not qualify to receive new licenses for the same
material under section 10.
9. Acceptance Not Required for Having Copies.
You are not required to accept this License in order to receive or
run a copy of the Program. Ancillary propagation of a covered work
occurring solely as a consequence of using peer-to-peer transmission
to receive a copy likewise does not require acceptance. However,
nothing other than this License grants you permission to propagate or
modify any covered work. These actions infringe copyright if you do
not accept this License. Therefore, by modifying or propagating a
covered work, you indicate your acceptance of this License to do so.
10. Automatic Licensing of Downstream Recipients.
Each time you convey a covered work, the recipient automatically
receives a license from the original licensors, to run, modify and
propagate that work, subject to this License. You are not responsible
for enforcing compliance by third parties with this License.
An "entity transaction" is a transaction transferring control of an
organization, or substantially all assets of one, or subdividing an
organization, or merging organizations. If propagation of a covered
work results from an entity transaction, each party to that
transaction who receives a copy of the work also receives whatever
licenses to the work the party's predecessor in interest had or could
give under the previous paragraph, plus a right to possession of the
Corresponding Source of the work from the predecessor in interest, if
the predecessor has it or can get it with reasonable efforts.
You may not impose any further restrictions on the exercise of the
rights granted or affirmed under this License. For example, you may
not impose a license fee, royalty, or other charge for exercise of
rights granted under this License, and you may not initiate litigation
(including a cross-claim or counterclaim in a lawsuit) alleging that
any patent claim is infringed by making, using, selling, offering for
sale, or importing the Program or any portion of it.
11. Patents.
A "contributor" is a copyright holder who authorizes use under this
License of the Program or a work on which the Program is based. The
work thus licensed is called the contributor's "contributor version".
A contributor's "essential patent claims" are all patent claims
owned or controlled by the contributor, whether already acquired or
hereafter acquired, that would be infringed by some manner, permitted
by this License, of making, using, or selling its contributor version,
but do not include claims that would be infringed only as a
consequence of further modification of the contributor version. For
purposes of this definition, "control" includes the right to grant
patent sublicenses in a manner consistent with the requirements of
this License.
Each contributor grants you a non-exclusive, worldwide, royalty-free
patent license under the contributor's essential patent claims, to
make, use, sell, offer for sale, import and otherwise run, modify and
propagate the contents of its contributor version.
In the following three paragraphs, a "patent license" is any express
agreement or commitment, however denominated, not to enforce a patent
(such as an express permission to practice a patent or covenant not to
sue for patent infringement). To "grant" such a patent license to a
party means to make such an agreement or commitment not to enforce a
patent against the party.
If you convey a covered work, knowingly relying on a patent license,
and the Corresponding Source of the work is not available for anyone
to copy, free of charge and under the terms of this License, through a
publicly available network server or other readily accessible means,
then you must either (1) cause the Corresponding Source to be so
available, or (2) arrange to deprive yourself of the benefit of the
patent license for this particular work, or (3) arrange, in a manner
consistent with the requirements of this License, to extend the patent
license to downstream recipients. "Knowingly relying" means you have
actual knowledge that, but for the patent license, your conveying the
covered work in a country, or your recipient's use of the covered work
in a country, would infringe one or more identifiable patents in that
country that you have reason to believe are valid.
If, pursuant to or in connection with a single transaction or
arrangement, you convey, or propagate by procuring conveyance of, a
covered work, and grant a patent license to some of the parties
receiving the covered work authorizing them to use, propagate, modify
or convey a specific copy of the covered work, then the patent license
you grant is automatically extended to all recipients of the covered
work and works based on it.
A patent license is "discriminatory" if it does not include within
the scope of its coverage, prohibits the exercise of, or is
conditioned on the non-exercise of one or more of the rights that are
specifically granted under this License. You may not convey a covered
work if you are a party to an arrangement with a third party that is
in the business of distributing software, under which you make payment
to the third party based on the extent of your activity of conveying
the work, and under which the third party grants, to any of the
parties who would receive the covered work from you, a discriminatory
patent license (a) in connection with copies of the covered work
conveyed by you (or copies made from those copies), or (b) primarily
for and in connection with specific products or compilations that
contain the covered work, unless you entered into that arrangement,
or that patent license was granted, prior to 28 March 2007.
Nothing in this License shall be construed as excluding or limiting
any implied license or other defenses to infringement that may
otherwise be available to you under applicable patent law.
12. No Surrender of Others' Freedom.
If conditions are imposed on you (whether by court order, agreement or
otherwise) that contradict the conditions of this License, they do not
excuse you from the conditions of this License. If you cannot convey a
covered work so as to satisfy simultaneously your obligations under this
License and any other pertinent obligations, then as a consequence you may
not convey it at all. For example, if you agree to terms that obligate you
to collect a royalty for further conveying from those to whom you convey
the Program, the only way you could satisfy both those terms and this
License would be to refrain entirely from conveying the Program.
13. Use with the GNU Affero General Public License.
Notwithstanding any other provision of this License, you have
permission to link or combine any covered work with a work licensed
under version 3 of the GNU Affero General Public License into a single
combined work, and to convey the resulting work. The terms of this
License will continue to apply to the part which is the covered work,
but the special requirements of the GNU Affero General Public License,
section 13, concerning interaction through a network will apply to the
combination as such.
14. Revised Versions of this License.
The Free Software Foundation may publish revised and/or new versions of
the GNU General Public License from time to time. Such new versions will
be similar in spirit to the present version, but may differ in detail to
address new problems or concerns.
Each version is given a distinguishing version number. If the
Program specifies that a certain numbered version of the GNU General
Public License "or any later version" applies to it, you have the
option of following the terms and conditions either of that numbered
version or of any later version published by the Free Software
Foundation. If the Program does not specify a version number of the
GNU General Public License, you may choose any version ever published
by the Free Software Foundation.
If the Program specifies that a proxy can decide which future
versions of the GNU General Public License can be used, that proxy's
public statement of acceptance of a version permanently authorizes you
to choose that version for the Program.
Later license versions may give you additional or different
permissions. However, no additional obligations are imposed on any
author or copyright holder as a result of your choosing to follow a
later version.
15. Disclaimer of Warranty.
THERE IS NO WARRANTY FOR THE PROGRAM, TO THE EXTENT PERMITTED BY
APPLICABLE LAW. EXCEPT WHEN OTHERWISE STATED IN WRITING THE COPYRIGHT
HOLDERS AND/OR OTHER PARTIES PROVIDE THE PROGRAM "AS IS" WITHOUT WARRANTY
OF ANY KIND, EITHER EXPRESSED OR IMPLIED, INCLUDING, BUT NOT LIMITED TO,
THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
PURPOSE. THE ENTIRE RISK AS TO THE QUALITY AND PERFORMANCE OF THE PROGRAM
IS WITH YOU. SHOULD THE PROGRAM PROVE DEFECTIVE, YOU ASSUME THE COST OF
ALL NECESSARY SERVICING, REPAIR OR CORRECTION.
16. Limitation of Liability.
IN NO EVENT UNLESS REQUIRED BY APPLICABLE LAW OR AGREED TO IN WRITING
WILL ANY COPYRIGHT HOLDER, OR ANY OTHER PARTY WHO MODIFIES AND/OR CONVEYS
THE PROGRAM AS PERMITTED ABOVE, BE LIABLE TO YOU FOR DAMAGES, INCLUDING ANY
GENERAL, SPECIAL, INCIDENTAL OR CONSEQUENTIAL DAMAGES ARISING OUT OF THE
USE OR INABILITY TO USE THE PROGRAM (INCLUDING BUT NOT LIMITED TO LOSS OF
DATA OR DATA BEING RENDERED INACCURATE OR LOSSES SUSTAINED BY YOU OR THIRD
PARTIES OR A FAILURE OF THE PROGRAM TO OPERATE WITH ANY OTHER PROGRAMS),
EVEN IF SUCH HOLDER OR OTHER PARTY HAS BEEN ADVISED OF THE POSSIBILITY OF
SUCH DAMAGES.
17. Interpretation of Sections 15 and 16.
If the disclaimer of warranty and limitation of liability provided
above cannot be given local legal effect according to their terms,
reviewing courts shall apply local law that most closely approximates
an absolute waiver of all civil liability in connection with the
Program, unless a warranty or assumption of liability accompanies a
copy of the Program in return for a fee.
END OF TERMS AND CONDITIONS
How to Apply These Terms to Your New Programs
If you develop a new program, and you want it to be of the greatest
possible use to the public, the best way to achieve this is to make it
free software which everyone can redistribute and change under these terms.
To do so, attach the following notices to the program. It is safest
to attach them to the start of each source file to most effectively
state the exclusion of warranty; and each file should have at least
the "copyright" line and a pointer to where the full notice is found.
<one line to give the program's name and a brief idea of what it does.>
Copyright (C) <year> <name of author>
This program is free software: you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation, either version 3 of the License, or
(at your option) any later version.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with this program. If not, see <http://www.gnu.org/licenses/>.
Also add information on how to contact you by electronic and paper mail.
If the program does terminal interaction, make it output a short
notice like this when it starts in an interactive mode:
<program> Copyright (C) <year> <name of author>
This program comes with ABSOLUTELY NO WARRANTY; for details type `show w'.
This is free software, and you are welcome to redistribute it
under certain conditions; type `show c' for details.
The hypothetical commands `show w' and `show c' should show the appropriate
parts of the General Public License. Of course, your program's commands
might be different; for a GUI interface, you would use an "about box".
You should also get your employer (if you work as a programmer) or school,
if any, to sign a "copyright disclaimer" for the program, if necessary.
For more information on this, and how to apply and follow the GNU GPL, see
<http://www.gnu.org/licenses/>.
The GNU General Public License does not permit incorporating your program
into proprietary programs. If your program is a subroutine library, you
may consider it more useful to permit linking proprietary applications with
the library. If this is what you want to do, use the GNU Lesser General
Public License instead of this License. But first, please read
<http://www.gnu.org/philosophy/why-not-lgpl.html>.
Makefile.am
+2
View File
@@ -0,0 +1,2 @@
ACLOCAL_AMFLAGS = -I m4
SUBDIRS = src man traces scripts
README.md
+134 -1
View File
@@ -1 +1,134 @@
# bvc-mahimahi
# bvc-mahimahi
bvc-mahimahi is a network emulation tools implemented by BVC (Bilibili Video Cloud team). The project is based on MIT mahimahi(http://mahimahi.mit.edu/). In order to support experiment using multiple HTTP protocols such as HTTP2 and HTTP3 (QUIC), BVC replaced the apache server with nginx server on the basis of the original implementation. And by dynamically configuring HTTP network modules in the nginx configuration file, it can support the recording of various websites and replaying the recorded traffic over multiple network protocols, so that will facilitate the developers to do measurement test and tune their web service performance by using the toolkit.
### Features
- Record and replay HTTP traffic
- Emulate numerous network conditions
- Use unmodified client applications
## Getting Started
### Build
```bash
./autogen.sh
./configure
make
sudo make install
```
### Dependencies
| Name | Typical package |
| ------ | ------ |
| Protocol Buffers | protobuf-compiler, libprotobuf-dev |
| autotools | autotools-dev |
| autoreconf | dh-autoreconf |
| iptables | iptables |
| pkg-config | pkg-config |
| dnsmasq | dnsmasq-base |
| debhelper (>= 9) | debhelper |
| OpenSSL | libssl-dev, ssl-cert |
| xcb | libxcb-present-dev |
| Cairo | libcairo2-dev |
| Pango | libpango1.0-dev |
| nginx | tengine-2.3.2 |
| spawn-fcgi | spawn-fcgi-1.6.4 |
| flup | flup-1.0.2 |
| fastcgi | fcgi2 |
### Play examples
- To record traffic from www.bilibili.com in Chromium:
```bash
cd mahimahi
mm-webrecord test
/path-to/Chromium --ignore-certificate-errors --ignore-ssl-errors https://www.bilibili.com
```
Then you will enter bilibili homepage. You can look through the web pages and click any interested item like video. The toolkit will record all the web HTTP traffic during the whole process until you type ‘exit’ to quit the shell.
```bash
exit
```
- Pre-works for web replay:
Before excuting the replay binary, you should firstly compile and install nginx with its network modules by configuration. In the example, the nginx version that we use is tengine-2.3.2, and the nginx QUIC network module is implemented by our team.
```bash
cd tengine-2.3.2
./configure --prefix=/data/tools/nginx --with-http_ssl_module --with-http_v2_module --add-dynamic-module=/path-to/nginx-quic-module
make && make install
```
You should also create a local root certificate authority (CA) for traffic replaying over HTTP2 protocol.
Create the root CA directory:
```bash
mkdir /data/CA
cd /data/CA
mkdir certs crl newcerts private
chmod 700 private
touch index.txt
echo 1000 > serial.txt
```
Generate the private key of the root CA:
```bash
openssl genrsa -aes256 -out /data/CA/private/ca.key.pem 4096
Enter pass phrase for ca.key.pem:SECRET
Verifying - Enter pass phrase for ca.key.pem:SECRET
chmod 400 /data/CA/private/ca.key.pem
```
Create the root CA config file:
```bash
cd /data/CA
cp /etc/ssl/openssl.cnf root_CA.cnf
vim root_CA.cnf
```
Edit CA config template to ensure the following fields to be properly set:
```bash
[ CA_default ]
dir = /path-to/mahimahi/CA
certs = $dir/certs
private = $dir/private
certificate = $certs/ca.cert.pem
private_key = $private/ca.key.pem
[ usr_cert ]
basicConstraints=CA:FALSE
keyUsage = nonRepudiation, digitalSignature, keyEncipherment
nsComment = "OpenSSL Generated Certificate"
subjectKeyIdentifier=hash
authorityKeyIdentifier=keyid,issuer
[ v3_ca ]
subjectKeyIdentifier=hash
authorityKeyIdentifier=keyid:always,issuer
basicConstraints = CA:true
keyUsage = cRLSign, keyCertSign
```
Generate the self-signed root CA certificate:
```bash
cd /data/CA
openssl req -new -x509 -days 3650 -key private/ca.key.pem -sha256 -extensions v3_ca -out certs/ca.cert.pem -config root_CA.cnf
```
The root CA certificate will be generated and saved in certs/ directory, and will be later used to generate self-signed certificates for each individual web server.
- To replay traffic over HTTP2 protocol in Chromium:
Run webreplay binary and input secret in order to generate certificates to be used by nginx server.
```bash
cd mahimahi
mm-webreplay test
```
Install self-signed root CA to Chromium:
Open Chromium → settings → Privacy and security → security → Manage certificates → Authorities → import /data/CA/cert/ca.cert.pem → restart Chromium
```bash
/path-to/Chromium https://www.bilibili.com
```
- To replay traffic over QUIC protocol in Chromium:
Because Chromium handles QUIC and TLS (over TCP) differently in cert verification, we should explicitly set some flags to force traffic run over QUIC. Especially, we shoud explicitly set hostname and QUIC version for those servers we expect them to server over QUIC, while the rest will serve over HTTP2.
```bash
/path-to/Chromium --quic-host-whitelist=<sitename> --origin-to-force-quic-on=<sitename:port> --quic-version=h3-Q050 https://www.bilibili.com
```
autogen.sh
Executable
+3
View File
@@ -0,0 +1,3 @@
#!/bin/sh
exec autoreconf -fi
build-package.sh
Executable
+3
View File
@@ -0,0 +1,3 @@
#!/bin/sh
gbp buildpackage --git-upstream-branch=master --git-upstream-tree=branch
configure.ac
+131
View File
@@ -0,0 +1,131 @@
# -*- Autoconf -*-
# Process this file with autoconf to produce a configure script.
AC_PREREQ([2.68])
AC_INIT([mahimahi], [0.98], [mahimahi@mit.edu])
AM_INIT_AUTOMAKE([foreign])
AC_CONFIG_SRCDIR([src/frontend/delayshell.cc])
AC_CONFIG_HEADERS([config.h])
LT_INIT
AC_CONFIG_MACRO_DIR([m4])
# Add picky CXXFLAGS
CXX11_FLAGS="-std=c++11 -pthread"
PICKY_CXXFLAGS="-pedantic -Wall -Wextra -Weffc++ -Werror"
AC_SUBST([CXX11_FLAGS])
AC_SUBST([PICKY_CXXFLAGS])
# Checks for programs.
AC_PROG_CXX
AC_ARG_VAR([IPTABLES], [path to iptables])
AC_PATH_PROG([IPTABLES], [iptables], [no], [$PATH$PATH_SEPARATOR/sbin$PATH_SEPARATOR/usr/sbin$PATH_SEPARATOR/bin$PATH_SEPARATOR/usr/bin])
if test "$IPTABLES" = "no"; then
AC_MSG_ERROR([iptables is required])
fi
AC_DEFINE_UNQUOTED([IPTABLES], ["$IPTABLES"], [path to iptables])
AC_ARG_VAR([IP], [path to ip])
AC_PATH_PROG([IP], [ip], [no], [$PATH$PATH_SEPARATOR/sbin$PATH_SEPARATOR/usr/sbin$PATH_SEPARATOR/bin$PATH_SEPARATOR/usr/bin])
if test "$IP" = "no"; then
AC_MSG_ERROR([ip is required])
fi
AC_DEFINE_UNQUOTED([IP], ["$IP"], [path to ip])
AC_ARG_VAR([APACHE2], [path to apache2])
AC_PATH_PROGS([APACHE2], [apache2 httpd], [no], [$PATH$PATH_SEPARATOR/sbin$PATH_SEPARATOR/usr/sbin$PATH_SEPARATOR/bin$PATH_SEPARATOR/usr/bin])
if test "$APACHE2" = "no"; then
AC_MSG_ERROR([apache2 is required])
fi
AC_DEFINE_UNQUOTED([APACHE2], ["$APACHE2"], [path to apache2])
AC_ARG_VAR([DNSMASQ], [path to dnsmasq])
AC_PATH_PROG([DNSMASQ], [dnsmasq], [no], [$PATH$PATH_SEPARATOR/sbin$PATH_SEPARATOR/usr/sbin$PATH_SEPARATOR/bin$PATH_SEPARATOR/usr/bin])
if test "$DNSMASQ" = "no"; then
AC_MSG_ERROR([dnsmasq is required])
fi
AC_DEFINE_UNQUOTED([DNSMASQ], ["$DNSMASQ"], [path to dnsmasq])
AC_PATH_PROG([PROTOC], [protoc], [])
AS_IF([test x"$PROTOC" = x],
[AC_MSG_ERROR([cannot find protoc, the Protocol Buffers compiler])])
AC_PATH_PROG([APXS], [apxs], [])
AS_IF([test x"$APXS" = x],
[AC_MSG_ERROR([cannot find apxs, the Apache extension tool])])
AC_SUBST([APACHE2_MODULE_DIR], [$(apxs -q exp_libexecdir)])
AC_SUBST([APACHE2_INCLUDE], [$(apxs -q exp_includedir)])
# Checks for apache configuration modules
AC_CHECK_FILE([$APACHE2_MODULE_DIR/mod_mpm_prefork.so],
[AC_DEFINE_UNQUOTED([MOD_MPM_PREFORK], ["$APACHE2_MODULE_DIR/mod_mpm_prefork.so"], [mod_prefork.so])],
[AC_MSG_ERROR([apache2 mod_mpm_prefork.so is required])])
AC_CHECK_FILE([$APACHE2_MODULE_DIR/mod_authz_core.so],
[AC_DEFINE_UNQUOTED([MOD_AUTHZ_CORE], ["$APACHE2_MODULE_DIR/mod_authz_core.so"], [mod_authz_core.so])],
[AC_MSG_ERROR([apache2 mod_authz_core.so is required])])
AC_CHECK_FILE([$APACHE2_MODULE_DIR/mod_ssl.so],
[AC_DEFINE_UNQUOTED([MOD_SSL], ["$APACHE2_MODULE_DIR/mod_ssl.so"], [mod_ssl.so])],
[AC_MSG_ERROR([apache2 mod_ssl.so is required])])
# Checks for SSL certificate file
AC_CHECK_FILE([/etc/ssl/certs/ssl-cert-snakeoil.pem], [],
[AC_MSG_ERROR([ssl certificate file is required])])
AC_DEFINE_UNQUOTED([MOD_SSL_CERTIFICATE_FILE], ["/etc/ssl/certs/ssl-cert-snakeoil.pem"], [path to ssl certificate file])
AC_DEFINE_UNQUOTED([MOD_SSL_KEY], ["/etc/ssl/private/ssl-cert-snakeoil.key"], [path to ssl key])
# Set path to mm-replayserver script
if test "${prefix}" = "NONE"; then
prefix=$ac_default_prefix
fi
AC_DEFINE_UNQUOTED([REPLAYSERVER], ["${prefix}/bin/mm-replayserver"], [path to mm-replayserver])
AC_DEFINE_UNQUOTED([MOD_DEEPCGI], ["${prefix}/lib/${DEB_HOST_MULTIARCH}/libmod_deepcgi.so"], [path to apache2 mod_deepcgi])
# Checks for libraries.
PKG_CHECK_MODULES([protobuf], [protobuf])
PKG_CHECK_MODULES([libssl], [libcrypto libssl])
PKG_CHECK_MODULES([libapr1], [apr-1])
PKG_CHECK_MODULES([XCB], [xcb])
PKG_CHECK_MODULES([XCBPRESENT], [xcb-present])
PKG_CHECK_MODULES([PANGOCAIRO], [pangocairo])
# Checks for header files.
AC_HEADER_RESOLV
AC_CHECK_HEADERS([arpa/inet.h fcntl.h netdb.h netinet/in.h paths.h sys/ioctl.h sys/socket.h unistd.h], [], [AC_MSG_ERROR([Missing required header file.])])
# Checks for typedefs, structures, and compiler characteristics.
AC_CHECK_HEADER_STDBOOL
AC_TYPE_UID_T
AC_C_INLINE
AC_TYPE_PID_T
AC_TYPE_SIZE_T
AC_TYPE_SSIZE_T
AC_TYPE_UINT16_T
AC_TYPE_UINT32_T
AC_TYPE_UINT64_T
AC_TYPE_UINT8_T
# Checks for library functions.
AC_CHECK_FUNCS([clock_gettime inet_ntoa memset mkdir setenv socket strerror strtol])
AC_CONFIG_FILES([Makefile
src/Makefile
src/util/Makefile
src/packet/Makefile
src/graphing/Makefile
src/http/Makefile
src/httpserver/Makefile
src/frontend/Makefile
src/protobufs/Makefile
src/tests/Makefile
man/Makefile
traces/Makefile
scripts/Makefile])
AC_OUTPUT
debian/changelog
Vendored
+67
View File
@@ -0,0 +1,67 @@
mahimahi (0.98-1) unstable; urgency=medium
* Fix FTBFS on glibc 2.25 (Closes: #882623)
-- Keith Winstein <keithw@mit.edu> Mon, 04 Dec 2017 15:57:28 -0800
mahimahi (0.97-1) unstable; urgency=medium
* Add script to convert "rates over time" to an mm-link trace
* Fix FTBFS on GCC 7 (Closes: #853534)
-- Keith Winstein <keithw@mit.edu> Wed, 23 Aug 2017 17:27:15 -0700
mahimahi (0.96-1) unstable; urgency=medium
* Fix error from deprecated TLS method and -Werror (Closes: #847398)
* Add new cellular trace (ATT-LTE-driving-2016)
-- Keith Winstein <keithw@cs.stanford.edu> Mon, 19 Dec 2016 17:37:39 -0800
mahimahi (0.95-1) unstable; urgency=medium
* Fix FTBFS against libxcb 1.12 (Closes: #839749)
-- Keith Winstein <keithw@mit.edu> Wed, 12 Oct 2016 02:55:25 -0400
mahimahi (0.94-1) unstable; urgency=medium
* Add CoDel and PIE implementations (Joseph Beshay)
-- Keith Winstein <keithw@mit.edu> Wed, 12 Oct 2016 01:03:27 -0400
mahimahi (0.93-1) unstable; urgency=medium
* Fix FTBFS with gcc 6
* Fix broken TCP DNS resolution
* Update mm-delay-graph to work with gnuplot 5
* Add new (2016) T-Mobile network trace to trace library
-- Keith Winstein <keithw@mit.edu> Thu, 08 Sep 2016 22:43:46 -0400
mahimahi (0.92-1) unstable; urgency=low
* Fix bug when /proc/net/route is empty
* Fix bug preventing HTTPS resources from working in mm-webreplay
* mm-link --once now quits only when both uplink and downlink have
-- Keith Winstein <keithw@mit.edu> Sat, 16 Jan 2016 18:47:17 -0800
mahimahi (0.91a-1) unstable; urgency=low
* Fix Debian build failure on some 64-bit platforms (#806368)
* Fix Debian build failure for source-only uploads (#806072)
-- Keith Winstein <keithw@mit.edu> Thu, 26 Nov 2015 11:35:47 -0800
mahimahi (0.91-1) unstable; urgency=low
* mm-webreplay now supports persistent HTTP connections (Deepak Narayanan)
-- Keith Winstein <keithw@mit.edu> Sat, 17 Oct 2015 05:47:59 -0700
mahimahi (0.90-1) unstable; urgency=low
* Initial release. (Closes: #782413)
-- Keith Winstein <keithw@mit.edu> Tue, 21 Apr 2015 00:48:16 -0700
debian/compat
Vendored
+1
View File
@@ -0,0 +1 @@
9
debian/control
Vendored
+42
View File
@@ -0,0 +1,42 @@
Source: mahimahi
Section: net
Priority: optional
Maintainer: Keith Winstein <keithw@mit.edu>
Homepage: http://mahimahi.mit.edu
Build-Depends: debhelper (>= 9), autotools-dev, dh-autoreconf, iptables, protobuf-compiler, libprotobuf-dev, pkg-config, libssl-dev, dnsmasq-base, ssl-cert, libxcb-present-dev, libcairo2-dev, libpango1.0-dev, iproute2, apache2-dev, apache2-bin
Standards-Version: 4.1.2.0
Vcs-Git: https://github.com/ravinet/mahimahi
Vcs-Browser: https://github.com/ravinet/mahimahi
Package: mahimahi
Architecture: any
Pre-Depends: ${misc:Pre-Depends}
Depends: ${shlibs:Depends}, ${misc:Depends}, iptables, dnsmasq-base, apache2-bin, gnuplot, iproute2, apache2-api-20120211
Recommends: mahimahi-traces
Description: tools for network emulation and analysis
Mahimahi is a suite of user-space tools for network emulation and analysis.
.
Each mahimahi tool spawns a lightweight container, generally connected
to the outside via a synthetic network device that observes packets in
transit or emulates a desired behavior.
.
The tools are composable so that a series of emulated network effects
can be chained together, with mahimahi containers nested inside each
other. Each tool takes an optional command to execute, so it is possible
to create a series of nested containers with one command line.
Package: mahimahi-traces
Architecture: all
Depends: ${misc:Depends}
Recommends: mahimahi
Description: network traces for the mahimahi toolkit
These trace files represent the time-varying capacity of U.S. cellular
networks as experienced by a mobile user. They were recorded using the
"Saturator" tool described in the research paper:
.
K. Winstein, A. Sivaraman, and H. Balakrishnan, "Stochastic Forecasts
Achieve High Throughput and Low Delay over Cellular Networks", USENIX
NSDI 2013.
.
The traces can be played back by the mm-link network emulator,
part of the mahimahi toolkit.
debian/copyright
Vendored
+36
View File
@@ -0,0 +1,36 @@
Format: http://www.debian.org/doc/packaging-manuals/copyright-format/1.0/
Upstream-Name: mahimahi
Source: https://github.com/keithw/mahimahi
Files: *
Copyright: 2013 The Mahimahi Authors <mahimahi@mit.edu>
License: GPL-3+
This program is free software: you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation, either version 3 of the License, or
(at your option) any later version.
.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
.
You should have received a copy of the GNU General Public License
along with this program. If not, see <http://www.gnu.org/licenses/>.
.
In addition, as a special exception, the copyright holders give
permission to link the code of portions of this program with the
OpenSSL library under certain conditions as described in each
individual source file, and distribute linked combinations including
the two.
.
You must obey the GNU General Public License in all respects for all
of the code used other than OpenSSL. If you modify file(s) with this
exception, you may extend this exception to your version of the
file(s), but you are not obligated to do so. If you do not wish to do
so, delete this exception statement from your version. If you delete
this exception statement from all source files in the program, then
also delete it here.
.
On Debian systems, the complete text of the GNU General
Public License version 3 can be found in "/usr/share/common-licenses/GPL-3".
debian/docs
Vendored
+1
View File
@@ -0,0 +1 @@
README.md
debian/mahimahi-traces.install
Vendored
+1
View File
@@ -0,0 +1 @@
usr/share/mahimahi/traces
debian/mahimahi.install
Vendored
+3
View File
@@ -0,0 +1,3 @@
usr/bin
usr/share/man
usr/lib
debian/mahimahi.lintian-overrides
Vendored
+22
View File
@@ -0,0 +1,22 @@
mahimahi binary: setuid-binary usr/bin/mm-delay 4755 root/root
mahimahi binary: setuid-binary usr/bin/mm-loss 4755 root/root
mahimahi binary: setuid-binary usr/bin/mm-onoff 4755 root/root
mahimahi binary: setuid-binary usr/bin/mm-webrecord 4755 root/root
mahimahi binary: setuid-binary usr/bin/mm-webreplay 4755 root/root
mahimahi binary: setuid-binary usr/bin/mm-link 4755 root/root
mahimahi binary: setuid-binary usr/bin/mm-meter 4755 root/root
# mahimahi's shells need to be setuid root to run unshare()
# (to create a new network namespace / Linux container)
#
# These programs drop permissions before executing the user's command
# and run in a sanitized environment.
mahimahi binary: package-name-doesnt-match-sonames
# The deepcgi Apache module is loaded only when Apache is run by mm-webreplay.
# (not installed with dh-apache2 because it doesn't belong in the
# Apache modules directory for general use)
mahimahi [amd64] binary: non-dev-pkg-with-shlib-symlink usr/lib/x86_64-linux-gnu/libmod_deepcgi.so.0.0.0 usr/lib/x86_64-linux-gnu/libmod_deepcgi.so
mahimahi [i386] binary: non-dev-pkg-with-shlib-symlink usr/lib/i386-linux-gnu/libmod_deepcgi.so.0.0.0 usr/lib/i386-linux-gnu/libmod_deepcgi.so
# The deepcgi Apache module is used by mm-webreplay.
# The module is not intended for other developers to link against.
debian/mahimahi.symbols
Vendored
+5
View File
@@ -0,0 +1,5 @@
libmod_deepcgi.so.0 mahimahi #MINVER#
deepcgi_module@Base 0
deepcgi_set_recordingdir@Base 0
deepcgi_set_workingdir@Base 0
replayserver_filename@Base 0
debian/rules
Vendored Executable
+25
View File
@@ -0,0 +1,25 @@
#!/usr/bin/make -f
# -*- makefile -*-
# Sample debian/rules that uses debhelper.
# This file was originally written by Joey Hess and Craig Small.
# As a special exception, when this file is copied by dh-make into a
# dh-make output file, you may use that output file without restriction.
# This special exception was added by Craig Small in version 0.37 of dh-make.
# Uncomment this to turn on verbose mode.
#export DH_VERBOSE=1
export DEB_BUILD_MAINT_OPTIONS=hardening=+all
DEB_HOST_MULTIARCH ?= $(shell dpkg-architecture -qDEB_HOST_MULTIARCH)
%:
dh $@ --with autoreconf
override_dh_fixperms-arch:
dh_fixperms
chmod 4755 debian/mahimahi/usr/bin/mm-delay
chmod 4755 debian/mahimahi/usr/bin/mm-loss
chmod 4755 debian/mahimahi/usr/bin/mm-onoff
chmod 4755 debian/mahimahi/usr/bin/mm-webrecord
chmod 4755 debian/mahimahi/usr/bin/mm-webreplay
chmod 4755 debian/mahimahi/usr/bin/mm-link
chmod 4755 debian/mahimahi/usr/bin/mm-meter
debian/source/format
Vendored
+1
View File
@@ -0,0 +1 @@
3.0 (quilt)
debian/watch
Vendored
+2
View File
@@ -0,0 +1,2 @@
version=3
https://github.com/ravinet/mahimahi/tags .*/mahimahi-(\d[\d\.]+)\.tar\.gz
fcgi/FcgiHandler.py
Executable
+18
View File
@@ -0,0 +1,18 @@
#!/usr/bin/env python
# -*- coding: UTF-8 -*-
from cgi import escape
import os
from flup.server.fcgi import WSGIServer
#from flup.server.fcgi_single import WSGIServer
from ReplayApp import ReplayApp
app = ReplayApp()
#WSGIServer(app,multiplexed=False,multithreaded=False,minSpare=2, maxSpare=12).run()
# note to myself: this setting is fucking important for interleaved push.
# any sort of mulitithreading/high concurrency will FUCK UP the push order!
# Never ever use multithreaded WSGI otherwise custom scheduler wont work
WSGIServer(app,multiplexed=True,multithreaded=False).run()
fcgi/ReplayApp.py
+327
View File
@@ -0,0 +1,327 @@
from StringIO import StringIO
from cgi import escape
import sys
import json
import os
import subprocess
from strip_chunked import decode
from mimetools import Message
from urlparse import urlparse
import random
import zlib
import string
class ReplayApp:
def _parse_push_strategy(self):
if self.push_strategy_file == "noop":
return
with open(self.push_strategy_file) as fd:
parsed_push_strategy = json.load(fd)
if len(parsed_push_strategy['push_configs']) > 0:
for push_strategy_for_host in parsed_push_strategy['push_configs']:
if 'push_host' in push_strategy_for_host.keys():
self.push_host.append(push_strategy_for_host['push_host'])
self.push_trigger_path.append(push_strategy_for_host['push_trigger'])
self.push_assets.append(push_strategy_for_host['push_resources'])
if 'hint_host' in push_strategy_for_host.keys():
self.hint_host.append(push_strategy_for_host['hint_host'])
self.hint_trigger_path.append(push_strategy_for_host['hint_trigger'])
self.hint_assets.append(push_strategy_for_host['hint_resources'])
self.hint_mimetype.append(push_strategy_for_host['hint_mimetype'])
mimetypes = []
for i,mimetype in enumerate(push_strategy_for_host['hint_mimetype']):
if 'image' in mimetype:
mimetypes.append('image')
elif 'css' in mimetype:
mimetypes.append('style')
elif 'javascript' in mimetype:
mimetypes.append('script')
elif 'font' in mimetype:
mimetypes.append('font')
else: #xhr
print "WARNING: unmatched mimetype",mimetype, push_strategy_for_host['hint_resources'][i]
mimetypes.append('')
self.hint_as_string.append(mimetypes)
def _query_replayserver(self, env, hostname, path):
passed_env = dict()
# remap for compat with replayserver
passed_env['SERVER_PROTOCOL'] = "HTTP/1.1"
passed_env['MAHIMAHI_CHDIR'] = env['WORKING_DIR']
passed_env['MAHIMAHI_RECORD_PATH'] = env['RECORDING_DIR']
passed_env['REQUEST_METHOD'] = 'GET' # we assume this is always get..
passed_env['HTTPS'] = "1"
passed_env['REQUEST_URI'] = path
passed_env['HTTP_HOST'] = hostname
if 'HTTP_RANGE' in env.keys():
passed_env['HTTP_RANGE'] = env['HTTP_RANGE']
p = subprocess.Popen([env['REPLAYSERVER_FN']], stdout=subprocess.PIPE, stderr=subprocess.PIPE, env=passed_env)
(replay_stdout, _) = p.communicate()
return replay_stdout
def _init_push_responsecache(self):
env = os.environ
for i,host in enumerate(self.push_host):
self.push_cache.append([])
for j,asset_name in enumerate(self.push_assets[i]):
parsed = urlparse(asset_name)
stripped = asset_name[len('https://'+parsed.netloc):]
asset_name = stripped
passed_host = parsed.netloc
# print "Caching... ", asset_name, parsed
# load responses into cache
self.push_cache[i].append(self._query_replayserver(env,passed_host,asset_name))
def _init_custom_scheduler(self):
#check if this instance of the replay app is running a custom scheduler
env = os.environ
if env['NGINX_CUSTOM_SCHEDULER'] == "1" and self.push_strategy_file != "noop":
self.has_custom_strategy = True
self.custom_strategy_host_trigger = env['NGINX_CUSTOM_TRIGGER_HOST']
self.custom_strategy_path_trigger = env['NGINX_CUSTOM_TRIGGER_PATH']
self.custom_strategy_offset_html = int(env['NGINX_CUSTOM_OFFSET'])
custom_strategy_html_content_file = os.path.join(os.path.dirname(self.push_strategy_file), env['NGINX_CUSTOM_CONTENT_FILE'])
print "Custom Replay Scheduler active for "+env['NGINX_CUSTOM_TRIGGER_HOST']
replay_response = self._query_replayserver(env,self.custom_strategy_host_trigger,self.custom_strategy_path_trigger)
self.custom_strategy_response = replay_response
c = zlib.compressobj()
with open(custom_strategy_html_content_file,'r') as fd:
decompressed_index = fd.read()
print decompressed_index[:self.custom_strategy_offset_html]
#print decompressed_index
self.custom_html_content_A = c.compress(decompressed_index[:self.custom_strategy_offset_html]) + c.flush(zlib.Z_SYNC_FLUSH)
self.custom_html_content_B = c.compress(decompressed_index[self.custom_strategy_offset_html:]) + c.flush(zlib.Z_FINISH)
self.CRITICAL_CSS_PATH = '/push__critical.css'
critical_css_file = os.path.join(os.path.dirname(self.push_strategy_file), env['NGINX_CRITICAL_CSS_FILE'])
print "Reading", critical_css_file
c = zlib.compressobj()
with open(critical_css_file,'r') as fd:
self.CRITICAL_CSS_CONTENT = c.compress(fd.read()) + c.flush(zlib.Z_FINISH)
print "Done Reading"
def __init__(self):
self.has_custom_strategy = False
self.custom_strategy_host_trigger = ''
self.custom_strategy_path_trigger = ''
self.custom_html_content_A = ''
self.custom_html_content_B = ''
self.custom_strategy_offset_html = 0
self.push_strategy_file = os.environ['PUSH_STRATEGY_FILE']
self.push_host = []
self.push_trigger_path = []
self.push_assets = []
self.push_cache = []
self.hint_host = []
self.hint_trigger_path = []
self.hint_assets = []
self.hint_as_string = []
self.hint_mimetype = []
self._parse_push_strategy()
self._init_push_responsecache()
self._init_custom_scheduler()
def __call__(self, environ, start_response):
#print "call! " + p['REQUEST_URI']
#print environ
hdrlist = []
env = dict(environ)
if env['HTTP_HOST'].endswith('.'):
env['HTTP_HOST'] = env['HTTP_HOST'][:-1]
#FastPath for Push Garbage:
if env['REQUEST_METHOD'] == "GET" and env['REQUEST_URI'].startswith('/__GARBAGE__'):
amount_kb = int(env['REQUEST_URI'].split('__')[2])
N = amount_kb * 1024
response = ''.join(random.choice(string.ascii_uppercase + string.digits) for _ in range(N))
start_response('200 OK', [('Content-Type', 'text/plain')])
yield response
#FastPath for Push Critical CSS:
elif env['REQUEST_METHOD'] == "GET" and env['HTTP_HOST'] == self.custom_strategy_host_trigger and env['REQUEST_URI'].startswith(self.CRITICAL_CSS_PATH):
start_response("200 OK",
[
('Content-Type', 'text/css'),
('Content-Encoding', 'deflate'),
])
yield self.CRITICAL_CSS_CONTENT
else:
cached_response = None
is_push = False
if env['REQUEST_METHOD'] == "GET":
for u,host in enumerate(self.push_host):
if host == env['HTTP_HOST']:
for v,push_resource in enumerate(self.push_assets[u]):
parsed = urlparse(push_resource)
stripped = push_resource[len('https://'+parsed.netloc):]
push_resource = stripped
if push_resource == env['REQUEST_URI']:
#print "pushing from cache..."
is_push = True
cached_response = self.push_cache[u][v]
#print env['HTTP_HOST'], env['REQUEST_URI']
if cached_response is None:
# print environ['REQUEST_URI'], "Not cached!"
passed_env = dict()
# remap for compat with replayserver
passed_env['MAHIMAHI_CHDIR'] = env['WORKING_DIR']
passed_env['MAHIMAHI_RECORD_PATH'] = env['RECORDING_DIR']
passed_env['REQUEST_METHOD'] = env['REQUEST_METHOD']
passed_env['REQUEST_URI'] = env['REQUEST_URI']
# env['SERVER_PROTOCOL'], is currently a hack to find the corresponding
# h1 traces
passed_env['SERVER_PROTOCOL'] = "HTTP/1.1"
passed_env['HTTP_HOST'] = env['HTTP_HOST']
if 'HTTP_RANGE' in env.keys():
passed_env['HTTP_RANGE'] = env['HTTP_RANGE']
if env['wsgi.url_scheme'] == 'https':
passed_env['HTTPS'] = "1"
# shell=True,
p = subprocess.Popen(
[env['REPLAYSERVER_FN']], stdout=subprocess.PIPE, stderr=subprocess.PIPE, env=passed_env)
(cached_response, replay_stderr) = p.communicate()
custom_interleaving = False
# Check if we must deliver using a custom strategy:
if env['REQUEST_METHOD'] == "GET" and env['HTTP_HOST'] == self.custom_strategy_host_trigger and env['REQUEST_URI'] == self.custom_strategy_path_trigger:
custom_interleaving = True
#print "Size: " + str(len(self.custom_html_content_A))
if not custom_interleaving:
response_header, response_body = cached_response.split('\r\n\r\n', 1)
else:
response_header, _ = self.custom_strategy_response.split('\r\n\r\n', 1)
status_line, headers_alone = response_header.split('\r\n', 1)
splitted_status = status_line.split(' ')
# response_code = status_line[1]
status_cleaned = ' '.join(splitted_status[1:])
#print status_cleaned
headers = Message(StringIO(headers_alone))
#print env['REQUEST_METHOD'], env['REQUEST_URI']
is_chunked = False
corsfound = False
hdrlist = []
for key in headers.keys():
if key == "transfer-encoding" and 'chunked' in headers[key]:
is_chunked = True
else:
if custom_interleaving and key.lower() == 'content-encoding':
hdrlist.append((key.strip(), 'deflate'))
elif custom_interleaving and key.lower() == 'content-length':
continue
else:
#Filter out any link header the site might emit.
if key not in ['expires', 'date', 'last-modified','link', 'alt-svc','content-security-policy']:
hdrlist.append((key.strip(), headers[key]))
if key.lower() == 'access-control-allow-origin':
corsfound = True
if not corsfound:
# this is required for font hinting...
# note that we will not overwrite cors headers b/c
# in some xhr situations * is not sufficient
hdrlist.append(('Access-Control-Allow-Origin','*'))
if not is_push and env['SERVER_PROTOCOL'] == "HTTP/2":
for i, push_host_strategy in enumerate(self.push_host):
if passed_env['HTTP_HOST'] == push_host_strategy:
if passed_env['REQUEST_URI'] == self.push_trigger_path[i]:
linkstr = ''
# TODO (bewo): is there any limitation?
for asset in self.push_assets[i]:
if linkstr != '':
linkstr += ','
# print asset;
#print asset
linkstr += '<' + asset + '>; rel=preload'
hdrlist.append(('x-extrapush', str(linkstr)))
print 'WILL PUSH: ' ,len(self.push_assets[i]) #//, ('x-extrapush', str(linkstr))
break
if not is_push:
for i, hint_host_strategy in enumerate(self.hint_host):
if passed_env['HTTP_HOST'] == hint_host_strategy:
if passed_env['REQUEST_URI'] == self.hint_trigger_path[i]:
linkstr = ''
for j, asset in enumerate(self.hint_assets[i]):
if linkstr != '':
linkstr += ','
as_string = self.hint_as_string[i][j]
if as_string != '':
as_string='; as='+as_string+''
if self.hint_as_string[i][j] == "font":
as_string += ";crossorigin"
linkstr_to_append = '<'+asset + '>; rel=preload'+as_string+';type="'+self.hint_mimetype[i][j]+'"'
linkstr += linkstr_to_append
hdrlist.append(('link', str(linkstr)))
# print 'WILL HINT: ' ,len(self.hint_assets[i]) #//, ('x-extrapush', str(linkstr))
break
# print "start response! " + environ['HTTP_HOST'] + " - " + environ['REQUEST_URI']
if not custom_interleaving:
if is_chunked:
# print "will decode chunked"
start_response(status_cleaned, hdrlist)
# print "chunked"
for chunk in decode(StringIO(response_body)):
yield str(chunk)
else:
start_response(status_cleaned, hdrlist)
yield response_body
else:
#note that we already decode the chunk
start_response(status_cleaned, hdrlist)
yield self.custom_html_content_A
yield self.custom_html_content_B
#start_response('200 OK', [('Content-Type', 'text/html')])
# yield replay_stdout
fcgi/strip_chunked.py
+55
View File
@@ -0,0 +1,55 @@
import re
# skeleton from https://github.com/simon-engledew/python-chunks/blob/master/chunks/__init__.py
def _from_pattern(pattern, type, *args):
def coerce(value):
value = str(value)
match = pattern.search(value)
if match is not None:
return type(match.group(1), *args)
raise ValueError('unable to coerce "%s" into a %s' %
(value, type.__name__))
return coerce
to_megabytes = lambda n: n * 1024 * 1024
to_hex = _from_pattern(re.compile('([-+]?[0-9A-F]+)', re.IGNORECASE), int, 16)
def decode(fileobj, chunk_limit=to_megabytes(5)):
""" Removes chunked encoding from fileobj """
hexsizestr = ''
len_size_bytes = 0
while True:
firstbyte = fileobj.read(1)
if firstbyte == '\r':
nextbyte = fileobj.read(1)
assert nextbyte == '\n'
else:
len_size_bytes += 1
hexsizestr += firstbyte
if len_size_bytes > 100:
raise OverflowError('too many digits...')
continue
length = to_hex(hexsizestr)
hexsizestr = ''
len_size_bytes = 0
if length > chunk_limit:
raise OverflowError(
'invalid chunk size of "%d" requested, max is "%d"' % (length, chunk_limit))
value = fileobj.read(length)
assert len(value) == length
yield value
tail = fileobj.read(2)
if not tail:
raise ValueError('missing \\r\\n after chunk')
assert tail == '\r\n', 'unexpected characters "%s" after chunk' % tail
if not length:
return
kill_nginx_cgi.sh
Executable
+12
View File
@@ -0,0 +1,12 @@
pids=$(ps -ef |grep nginx | grep master | grep -v 'grep'| awk '{print $2}')
for pid in $pids
do
kill $pid
done
pids=$(ps -ef |grep fcgi | grep -v 'grep'|awk '{print $2}')
for pid in $pids
do
kill $pid
done
m4/.gitignore
+5
View File
@@ -0,0 +1,5 @@
/libtool.m4
/ltoptions.m4
/ltsugar.m4
/ltversion.m4
/lt~obsolete.m4
man/Makefile.am
+11
View File
@@ -0,0 +1,11 @@
dist_man_MANS = mahimahi.1 mm-link.1 mm-replayserver.1
# the rest just link to one of the above
dist_man_MANS += mm-delay.1
dist_man_MANS += mm-loss.1
dist_man_MANS += mm-onoff.1
dist_man_MANS += mm-throughput-graph.1
dist_man_MANS += mm-delay-graph.1
dist_man_MANS += mm-meter.1
dist_man_MANS += mm-webrecord.1
dist_man_MANS += mm-webreplay.1
man/mahimahi.1
+255
View File
@@ -0,0 +1,255 @@
.\" Hey, EMACS: -*- nroff -*-
.\" First parameter, NAME, should be all caps
.\" Second parameter, SECTION, should be 1-8, maybe w/ subsection
.\" other parameters are allowed: see man(7), man(1)
.TH MAHIMAHI 1 "March 2015"
.\" Please adjust this date whenever revising the manpage.
.\"
.\" Some roff macros, for reference:
.\" .nh disable hyphenation
.\" .hy enable hyphenation
.\" .ad l left justify
.\" .ad b justify to both left and right margins
.\" .nf disable filling
.\" .fi enable filling
.\" .br insert line break
.\" .sp <n> insert n+1 empty lines
.\" for manpage-specific macros, see man(7)
.SH NAME
\fBmahimahi\fP \- lightweight, composable network-emulation tools
link emulation: \fBmm-delay\fP, \fBmm-loss\fP, \fBmm-onoff\fP, \fBmm-link\fP
analysis scripts: \fBmm-throughput-graph\fP, \fBmm-delay-graph\fP
observation: \fBmm-meter\fP
record and replay multi-origin websites: \fBmm-webrecord\fP, \fBmm-webreplay\fP
.SH DESCRIPTION
\fBmahimahi\fP is a suite of user-space tools for network emulation and analysis.
Each mahimahi tool spawns a lightweight container, generally connected to the
outside via a synthetic network device that observes packets in
transit or emulates a desired behavior.
The tools are composable so that a series of emulated network effects
can be chained together, with mahimahi containers nested inside each
other. Each tool takes an optional command to execute, so it
is possible to create a series of nested containers with one command
line.
.SH LINK EMULATION TOOLS
.SY mm-delay
.I delay
.RI [ command... ]
.YS
.
.IP ""
.RS
Every packet is delayed by the specified
.I delay
(in milliseconds) entering and leaving the container.
.RE
.SY mm-loss
uplink|downlink
.I rate
.RI [ command... ]
.YS
.
.IP ""
.RS
Packets are lost at the given
.I rate
either when leaving (uplink) or entering (downlink) the container.
.I rate
is a number between 0 and 1.
.RE
.SY mm-onoff
uplink|downlink
.I mean-on-time
.I mean-off-time
.RI [ command... ]
.YS
.
.IP ""
.RS
The uplink or downlink will be
intermittent and will switch between connected and disconnected states
according to a Poisson point process with specified average durations
spent "on" and "off".
.RE
.SY mm-link
.OP --uplink-log=\fIfilename\fR
.OP --downlink-log=\fIfilename\fR
.OP --meter-uplink
.OP --meter-uplink-delay
.OP --meter-downlink
.OP --meter-downlink-delay
.OP --once
.I uplink-filename
.I downlink-filename
.RI [ command... ]
.YS
.SY mm-throughput-graph
.SY mm-delay-graph
.YS
.
.IP ""
.RS
Emulates a throughput-limited link with a specified packet-delivery schedule
and analyzes the resulting performance. See
.BR mm-link (1).
.RE
.SH OBSERVATION TOOLS
.SY mm-meter
.OP --meter-uplink
.OP --meter-downlink
.RI [ command... ]
.YS
.
.IP ""
.RS
Displays an animated live plot of the transfer rate entering or leaving the container.
.RE
.SH RECORD AND REPLAY WEBSITES
.SY mm-webrecord
.I directory
.RI [ command... ]
.YS
.
.IP ""
.RS
Transparently proxies outgoing HTTP and HTTPS connections, saving the
requests, corresponding responses, and IP address of each Web
server contacted in the given \fIdirectory\fR. \fBmm-webrecord\fP
uses a self-signed TLS certificate in its HTTPS proxy, causing typical
Web browsers to reject it. For testing or debugging purposes, this
behavior can usually be turned off, e.g.: with the
\fB--no-check-certificate\fP option to
.BR wget (1)
or the \fB--ignore-certificate-errors\fP option to
.BR chromium-browser (1).
.RE
.SY mm-webreplay
.I directory
.RI [ command... ]
.YS
.
.IP ""
.RS
Replays a saved session from a previous run of \fBmm-webrecord\fR.
Unlike most mahimahi tools, the \fBmm-webreplay\fP container
does not have a network connection to the outside world. Instead,
it has dummy network interfaces bound to each IP address on which a
Web server in the saved session had answered a request. \fPmm-webreplay\fR runs an
.BR apache2 (8)
Web server bound to each such IP address inside the container. Each
Web server emulates the corresponding server from the saved
session. When receiving a request that matches one in the \fIdirectory\fR, the
corresponding apache2 replies with the same reply as previously
captured.
\fBmm-webreplay\fP can be used to measure the performance of Web
browsers on complex websites and the effect of changes in Web
protocols (e.g. HTTP, HTTP/2, SPDY, QUIC). Unlike tools like web-page-replay,
\fBmm-webreplay\fP preserves the sharded structure of a website, binds to
the actual IP addresses that the real website used, and serves requests from
real Web servers.
.RE
.SH ENVIRONMENT
The MAHIMAHI_BASE environment variable is set to an IP address of the
host, outside any container. This can be used to conduct scripted
measurements over a series of mahimahi containers chained together.
.SH EXAMPLES
To spawn a shell with a delayed, lossy link to the Internet:
.IP ""
.RS
.EX
\fB$\fR mm-delay 50 mm-loss uplink 0.2
\fB[delay 50 ms] [loss up=0.1] $\fR
.EE
.RE
To run ping over the same link:
.IP ""
.RS
.EX
\fB$\fR mm\-delay 50 mm\-loss uplink 0.2 sh \-c 'ping \-c 10 \-n $MAHIMAHI_BASE'
PING 100.64.0.1 (100.64.0.1) 56(84) bytes of data.
64 bytes from 100.64.0.1: icmp_seq=1 ttl=63 time=101 ms
64 bytes from 100.64.0.1: icmp_seq=2 ttl=63 time=100 ms
64 bytes from 100.64.0.1: icmp_seq=4 ttl=63 time=101 ms
64 bytes from 100.64.0.1: icmp_seq=5 ttl=63 time=100 ms
64 bytes from 100.64.0.1: icmp_seq=7 ttl=63 time=101 ms
64 bytes from 100.64.0.1: icmp_seq=8 ttl=63 time=101 ms
64 bytes from 100.64.0.1: icmp_seq=9 ttl=63 time=101 ms
64 bytes from 100.64.0.1: icmp_seq=10 ttl=63 time=101 ms
--- 100.64.0.1 ping statistics ---
10 packets transmitted, 8 received, 20% packet loss, time 8999ms
rtt min/avg/max/mdev = 100.910/101.009/101.092/0.279 ms
.EE
.RE
To record a page load from \fIwww.nytimes.com\fR:
.IP ""
.RS
.EX
\fB$\fR mm\-webrecord /tmp/nytimes chromium-browser \-\-ignore\-certificate\-errors \-\-user\-data\-dir=/tmp/nonexistent$(date +%s%N) www.nytimes.com
.EE
The use of \fI\-\-user\-data\-dir=/tmp/nonexistent$(date +%s%N)\fR is to prevent the browser from reusing an existing chromium-browser process.
.RE
To make Chrome retrieve the saved website over a delayed, lossy link whose throughput is limited to 1 full-sized packet per millisecond:
.IP ""
.RS
.EX
\fB$\fR mm\-webreplay /tmp/nytimes mm\-delay 50 mm\-loss uplink 0.1 mm-link <(echo 1) <(echo 1) \-\- chromium-browser \-\-ignore\-certificate\-errors \-\-user\-data\-dir=/tmp/nonexistent$(date +%s%N) www.nytimes.com
.EE
.RE
To emulate a variable cellular network and visualize a process's use of the network:
.IP ""
.RS
.EX
\fB$\fR mm\-delay 20 mm\-link \-\-meter-all /usr/share/mahimahi/traces/Verizon-LTE-short.up /usr/share/mahimahi/traces/Verizon-LTE-short.down
\fB[delay 20 ms] [link] $\fR
.EE
.SH SEE ALSO
.BR mm-link (1)
Project home page:
.I http://mahimahi.mit.edu
.br
.SH AUTHOR
Mahimahi was written by Ravi Netravali, Anirudh Sivaraman, Greg D. Hill, Deepak Narayanan, and Keith Winstein.
.SH BUGS
Please report bugs to \fImahimahi@mit.edu\fP.
man/mm-delay-graph.1
+1
View File
@@ -0,0 +1 @@
.so man1/mm-link.1
man/mm-delay.1
+1
View File
@@ -0,0 +1 @@
.so man1/mahimahi.1
man/mm-link.1
+135
View File
@@ -0,0 +1,135 @@
.\" Hey, EMACS: -*- nroff -*-
.\" First parameter, NAME, should be all caps
.\" Second parameter, SECTION, should be 1-8, maybe w/ subsection
.\" other parameters are allowed: see man(7), man(1)
.TH mm-link 1 "October 2018"
.\" Please adjust this date whenever revising the manpage.
.\"
.\" Some roff macros, for reference:
.\" .nh disable hyphenation
.\" .hy enable hyphenation
.\" .ad l left justify
.\" .ad b justify to both left and right margins
.\" .nf disable filling
.\" .fi enable filling
.\" .br insert line break
.\" .sp <n> insert n+1 empty lines
.\" for manpage-specific macros, see man(7)
.SH NAME
mm-link - UNIX shell connected to an emulated link with a user-specified packet-delivery schedule.
.SH SYNOPSIS
.B mm-link
\fIuplink\fP
\fIdownlink\fP
[\-\- command...]
.br
.SH DESCRIPTION
mm-link is a network emulation tool that emulates links using packet delivery
trace files (\fIuplink\fP for the uplink direction and \fIdownlink\fP for the downlink direction) provided on the command
line. mm-link uses \fBclone\fP(2) to fork a new shell in a distinct network
namespace. uplink_trace_file emulates the link from mm-link to the Internet
and downlink_trace_file emulates the link from the Internet to mm-link.
mm-link can emulate both time-varying links, such as cellular links, and
links with fixed link speeds. When a packet arrives at the link (from either
the Internent or from mm-link), it is directly placed into one of two packet
queues depending on its intended direction: the uplink queue or the downlink
queue. mm-link releases packets from each queue based on the corresponding
input packet-delivery trace.
Each line in the trace represents a packet delivery opportunity: the time at
which an MTU-sized packet can be delivered in the emulation. Accounting is done
at the byte-level, and each delivery opportunity represents the ability to
deliver 1500 bytes. Thus, a single line in the trace file can delivery several
smaller packets whose sizes sum to 1500 bytes. Delivery opportunities are
wasted if bytes are unavailable at the instant of an opportunity. When
mm-link reaches the end of an input trace file, it wraps around to the
beginning of the trace file. mm-link can be nested within delayshell (1) to
flexibly create links with a user-supplied one-way delay and a user-supplied
link rate.
To exit mm-link, simply type "exit" or CTRL-D inside mm-link.
.SH OUTPUT
mm-link can optionally log detailed performance information for both the uplink and downlink, specified with the \fB--uplink-log\fR and \fB--downlink-log\fR flags respectively. A log file has the following format:
.EX
# mahimahi mm-link (name of link) [/path/to/trace] > /path/to/log
# command line: [command used to run mm-link]
# queue: [queue information]
# init timestamp: [start time of mm-link]
# base timestamp: [starting log timestamp]
[# mahimahi config: [mahimahi shell prefix]]
[log lines]
.EE
Each log line is one of the following:
[timestamp] # packet_size
.
.IP ""
.RS
An unused delivery opportunity.
.RE
[timestamp] + packet_size
.
.IP ""
.RS
A packet arrival to the link, it may be dropped later
.RE
[timestamp] - packet_size
.
.IP ""
.RS
A successful packet delivery
.RE
[timestamp] d packets_dropped bytes_dropped
.
.IP ""
.RS
A dropped packet (or multiple packets)
.RE
.SH EXAMPLE
.nf
To emulate a 12 Mbit/s link (in each direction), make a 12 Mbit/s
tracefile, "12Mbps_trace". This file can be of arbitrary length
and must follow the pattern below:
0
1
2
3
4
5...
The link above delivers an MTU-sized packet (1500 bytes or 12000 bits) every
ms.
Run mm-link with:
$ mm-link 12Mbps_trace 12Mbps_trace
All programs run from within mm-link are sent according
to the packet delivery times specified in 12Mbps_trace.
mm-link
[...] (copyright notice omitted)
.fi
.SH SEE ALSO
.BR mahimahi (1),
.BR mm-delay (1),
.BR mm-webrecord (1),
.BR mm-webreplay (1).
Project home page:
.I http://mahimahi.mit.edu
.br
.SH AUTHOR
Mahimahi was written by Ravi Netravali, Anirudh Sivaraman, Greg D. Hill, Deepak Narayanan, and Keith Winstein.
.SH BUGS
Please report bugs to \fImahimahi@mit.edu\fP.
man/mm-loss.1
+1
View File
@@ -0,0 +1 @@
.so man1/mahimahi.1
man/mm-meter.1
+1
View File
@@ -0,0 +1 @@
.so man1/mahimahi.1
man/mm-onoff.1
+1
View File
@@ -0,0 +1 @@
.so man1/mahimahi.1
man/mm-replayserver.1
+38
View File
@@ -0,0 +1,38 @@
.\" Hey, EMACS: -*- nroff -*-
.\" First parameter, NAME, should be all caps
.\" Second parameter, SECTION, should be 1-8, maybe w/ subsection
.\" other parameters are allowed: see man(7), man(1)
.TH MM-REPLAYSERVER 1 "March 2015"
.\" Please adjust this date whenever revising the manpage.
.\"
.\" Some roff macros, for reference:
.\" .nh disable hyphenation
.\" .hy enable hyphenation
.\" .ad l left justify
.\" .ad b justify to both left and right margins
.\" .nf disable filling
.\" .fi enable filling
.\" .br insert line break
.\" .sp <n> insert n+1 empty lines
.\" for manpage-specific macros, see man(7)
.SH NAME
\fBmm-replayserver\fP \- helper program used by
.BR mm-webreplay (1).
.SH DESCRIPTION
mm-replayserver is a program that replies to previously-saved
HTTP requests with the corresponding HTTP response. See
.BR mm-webreplay (1).
.SH SEE ALSO
.BR mahimahi (1)
Project home page:
.I http://mahimahi.mit.edu
.br
.SH AUTHOR
Mahimahi was written by Ravi Netravali, Anirudh Sivaraman, Greg D. Hill, Deepak Narayanan, and Keith Winstein.
.SH BUGS
Please report bugs to \fImahimahi@mit.edu\fP.
man/mm-throughput-graph.1
+1
View File
@@ -0,0 +1 @@
.so man1/mm-link.1
man/mm-webrecord.1
+1
View File
@@ -0,0 +1 @@
.so man1/mahimahi.1
man/mm-webreplay.1
+1
View File
@@ -0,0 +1 @@
.so man1/mahimahi.1
scripts/Makefile.am
+1
View File
@@ -0,0 +1 @@
dist_bin_SCRIPTS = mm-throughput-graph mm-delay-graph
scripts/mm-delay-graph
Executable
+157
View File
@@ -0,0 +1,157 @@
#!/usr/bin/env perl
use warnings;
use strict;
use POSIX;
sub usage
{
die qq{Usage: $0 [filename]\n};
}
if ( scalar @ARGV > 1 ) {
usage;
}
my $first_timestamp = undef;
my $last_timestamp = undef;
my $base_timestamp = undef;
my @delays;
my %signal_delay;
my $points;
LINE: while ( <> ) {
chomp;
if ( m{^# base timestamp: (\d+)} ) {
if ( defined $base_timestamp ) {
die "base timestamp multiply defined";
} else {
$base_timestamp = $1;
}
next LINE;
} elsif ( m{^#} ) {
next LINE;
}
# parse and validate line
my ( $timestamp, $event_type, $num_bytes, $delay ) = split /\s+/, $_;
if ( not defined $num_bytes ) {
die q{Format: timestamp event_type num_bytes [delay]};
}
if ( $timestamp !~ m{^\d+$} ) {
die qq{Invalid timestamp: $timestamp};
}
if ( $num_bytes !~ m{^\d+$} ) {
die qq{Invalid byte count: $num_bytes};
}
if ( not defined $base_timestamp ) {
die "logfile is missing base timestamp";
}
$timestamp -= $base_timestamp; # correct for startup time variation
if ( not defined $last_timestamp ) {
$last_timestamp = $first_timestamp = $timestamp;
}
$last_timestamp = max( $timestamp, $last_timestamp );
# process the event
if ( $event_type eq q{-} ) {
if ( not defined $delay ) {
die q{Departure format: timestamp - num_bytes delay};
}
push @delays, $delay;
$signal_delay{ $timestamp - $delay } = min( $delay, $signal_delay{ $timestamp - $delay } );
$points .= ($timestamp - $delay) / 1000.0 . " $delay\n";
}
}
sub max {
my $maxval = - POSIX::DBL_MAX;
for ( @_ ) {
next unless defined $_;
if ( $_ > $maxval ) {
$maxval = $_;
}
}
return $maxval;
}
sub min {
my $minval = POSIX::DBL_MAX;
for ( @_ ) {
next unless defined $_;
if ( $_ < $minval ) {
$minval = $_;
}
}
return $minval;
}
# calculate statistics
if ( scalar @delays == 0 ) {
die q{Must have at least one departure event};
}
@delays = sort { $a <=> $b } @delays;
my $pp95 = $delays[ 0.95 * scalar @delays ];
my $pp50 = $delays[ 0.5 * scalar @delays ];
# measure signal delay every millisecond
# = minimum time for a message created at time t to get to receiver
my @signal_delay_samples = sort { $a <=> $b } keys %signal_delay;
for ( my $ts = $signal_delay_samples[ -1 ]; $ts >= $signal_delay_samples[ 0 ]; $ts-- ) {
if ( not defined $signal_delay{ $ts } ) {
$signal_delay{ $ts } = $signal_delay{ $ts + 1 } + 1;
}
}
my $signal_delay_points;
for ( my $ts = $signal_delay_samples[ 0 ]; $ts <= $signal_delay_samples[ -1 ]; $ts++ ) {
$signal_delay_points .= $ts / 1000.0 . " " . $signal_delay{ $ts } . "\n";
}
my @signal_delays = sort { $a <=> $b } values %signal_delay;
my $pp95s = $signal_delays[ 0.95 * scalar @signal_delays ];
printf STDERR qq{95th percentile per-packet queueing delay: %.0f ms\n}, $pp95;
printf STDERR qq{95th percentile signal delay: %.0f ms\n}, $pp95s;
my $graph_max = $signal_delays[-1] * 1.2;
# make graph
open GNUPLOT, q{| gnuplot} or die;
my $lower_limit = min( 30, int $pp50 );
print GNUPLOT <<END;
set xlabel "time (s)"
set ylabel "delay (ms)
set logscale y
set yrange [$lower_limit:$graph_max]
set ytics add ($lower_limit)
set key center outside top horizontal
set terminal svg size 1024,560 fixed enhanced fname 'Arial' solid mouse standalone name "Delay"
set output "/dev/stdout"
END
printf GNUPLOT qq{plot [%f:%f] "-" using 1:2 title "Signal delay" with filledcurves above x1 lt 1 fs solid 0.4 noborder fc rgb "#0000ff", "-" using 1:2 title "Per-packet delay" with points lt 7 ps 0.15 lc rgb "#00ff0000", %f with lines lw 10 lc rgb "#800000ff" title "95th percentile signal delay (%.0f ms)", %f with lines lw 10 lc rgb "#80ff0000" title "95th percentile per-packet delay (%.0f ms)"\n}, $first_timestamp / 1000.0, $last_timestamp / 1000.0, $pp95s, $pp95s, $pp95, $pp95;
print GNUPLOT $signal_delay_points;
print GNUPLOT "e\n";
print GNUPLOT $points;
close GNUPLOT or die qq{$!};
scripts/mm-rate-to-events
Executable
+39
View File
@@ -0,0 +1,39 @@
#!/usr/bin/perl
# converts a trace FROM varying rate (in kilobits per millisecond,
# sampled once per millisecond)
# TO packet delivery occurrences (millisecond timestamps when an
# MTU-sized datagram, i.e. 12,000 bits, can be delivered)
#
# The latter is the format that mm-link expects.
#
# KJW 7/28/2017
use strict;
use warnings;
use Scalar::Util qw[looks_like_number];
use POSIX qw[ceil];
my $time = 0;
my $reserve_bits = 0;
my $PACKET_LENGTH = 12000; # bits
# input trace gives a number of kilobits that can be delivered each millisecond
while (my $kilobits_this_millisecond = <>) {
chomp $kilobits_this_millisecond;
die qq{Not a number: "$kilobits_this_millisecond"}
unless looks_like_number $kilobits_this_millisecond;
# add current rate to "reserve"
$reserve_bits += 1000 * $kilobits_this_millisecond;
# if reserve has enough bits for a packet, deliver it now
# output trace gives timestamps of packet delivery occurrences
while ( $reserve_bits >= $PACKET_LENGTH ) {
print qq{$time\n};
$reserve_bits -= $PACKET_LENGTH;
}
# advance time by one millisecond
$time++;
}
scripts/mm-throughput-graph
Executable
+224
View File
@@ -0,0 +1,224 @@
#!/usr/bin/env perl
use warnings;
use strict;
use POSIX;
sub usage
{
die qq{Usage: $0 MS_PER_BIN [filename]\n};
}
if ( scalar @ARGV < 1 or scalar @ARGV > 2 ) {
usage;
}
my $MS_PER_BIN = shift;
if ( $MS_PER_BIN !~ m{^\d+$} ) {
usage;
}
sub ms_to_bin {
return int( $_[0] / $MS_PER_BIN );
}
sub bin_to_seconds {
return sprintf q{%.3f}, $_[0] * $MS_PER_BIN / 1000.0;
}
my ( %capacity, %arrivals, %departures );
my $first_timestamp = undef;
my $last_timestamp = undef;
my $base_timestamp = undef;
my $capacity_sum;
my $arrival_sum;
my $departure_sum;
my @delays;
my %signal_delay;
LINE: while ( <> ) {
chomp;
if ( m{^# base timestamp: (\d+)} ) {
if ( defined $base_timestamp ) {
die "base timestamp multiply defined";
} else {
$base_timestamp = $1;
}
next LINE;
} elsif ( m{^#} ) {
next LINE;
}
# parse and validate line
my ( $timestamp, $event_type, $num_bytes, $delay ) = split /\s+/, $_;
if ( not defined $num_bytes ) {
die q{Format: timestamp event_type num_bytes [delay]};
}
if ( $timestamp !~ m{^\d+$} ) {
die qq{Invalid timestamp: $timestamp};
}
if ( $num_bytes !~ m{^\d+$} ) {
die qq{Invalid byte count: $num_bytes};
}
if ( not defined $base_timestamp ) {
die "logfile is missing base timestamp";
}
$timestamp -= $base_timestamp; # correct for startup time variation
if ( not defined $last_timestamp ) {
$last_timestamp = $first_timestamp = $timestamp;
}
$last_timestamp = max( $timestamp, $last_timestamp );
my $num_bits = $num_bytes * 8;
my $bin = ms_to_bin( $timestamp );
# process the event
if ( $event_type eq q{+} ) {
$arrivals{ $bin } += $num_bits;
$arrival_sum += $num_bits;
} elsif ( $event_type eq q{#} ) {
$capacity{ $bin } += $num_bits;
$capacity_sum += $num_bits;
} elsif ( $event_type eq q{-} ) {
if ( not defined $delay ) {
die q{Departure format: timestamp - num_bytes delay};
}
$departures{ $bin } += $num_bits;
if ( $delay < 0 ) {
die qq{Invalid delay: $delay};
}
if ( $timestamp - $delay < 0 ) {
die qq{Invalid timestamp and delay: ts=$timestamp, delay=$delay};
}
push @delays, $delay;
$departure_sum += $num_bits;
$signal_delay{ $timestamp - $delay } = min( $delay,
(defined $signal_delay{ $timestamp - $delay })
? $signal_delay{ $timestamp - $delay }
: POSIX::DBL_MAX );
} else {
die qq{Unknown event type: $event_type};
}
}
sub min {
my $minval = POSIX::DBL_MAX;
for ( @_ ) {
if ( $_ < $minval ) {
$minval = $_;
}
}
return $minval;
}
sub max {
my $maxval = - POSIX::DBL_MAX;
for ( @_ ) {
if ( $_ > $maxval ) {
$maxval = $_;
}
}
return $maxval;
}
if ( not defined $first_timestamp ) {
die q{Must have at least one event};
}
# calculate statistics
my $duration = ($last_timestamp - $first_timestamp) / 1000.0;
my $average_capacity = ($capacity_sum / $duration) / 1000000.0;
my $average_ingress = ($arrival_sum / $duration) / 1000000.0;
my $average_throughput = ($departure_sum / $duration) / 1000000.0;
if ( scalar @delays == 0 ) {
die q{Must have at least one departure event};
}
@delays = sort { $a <=> $b } @delays;
my $pp95 = $delays[ 0.95 * scalar @delays ];
# measure signal delay every millisecond
# = minimum time for a message created at time t to get to receiver
my @signal_delay_samples = sort { $a <=> $b } keys %signal_delay;
for ( my $ts = $signal_delay_samples[ -1 ]; $ts >= $signal_delay_samples[ 0 ]; $ts-- ) {
if ( not defined $signal_delay{ $ts } ) {
$signal_delay{ $ts } = $signal_delay{ $ts + 1 } + 1;
}
}
my @signal_delays = sort { $a <=> $b } values %signal_delay;
my $pp95s = $signal_delays[ 0.95 * scalar @signal_delays ];
printf STDERR qq{Average capacity: %.2f Mbits/s\n}, $average_capacity;
printf STDERR qq{Average throughput: %.2f Mbits/s (%.1f%% utilization)\n}, $average_throughput, 100.0 * $average_throughput / $average_capacity;
printf STDERR qq{95th percentile per-packet queueing delay: %.0f ms\n}, $pp95;
printf STDERR qq{95th percentile signal delay: %.0f ms\n}, $pp95s;
# make graph
my $earliest_bin = min( keys %arrivals, keys %capacity, keys %departures );
my $latest_bin = max( keys %arrivals, keys %capacity, keys %departures );
if ( $earliest_bin == $latest_bin ) {
die q{MS_PER_BIN is too large for length of trace};
}
my $current_buffer_occupancy = 0;
sub default {
return defined $_[ 0 ] ? $_[ 0 ] : 0;
}
open GNUPLOT, q{| gnuplot} or die;
print GNUPLOT <<END;
set xlabel "time (s)"
set ylabel "throughput (Mbits/s)"
set key center outside top horizontal
set style fill solid 0.2 noborder
set terminal svg size 1024,560 fixed fname 'Arial' rounded solid mouse standalone name "Throughput"
set output "/dev/stdout"
END
printf GNUPLOT qq{plot [%f:%f] "-" using 1:2 title "Capacity (mean %.2f Mbits/s)" with filledcurves above x1 lw 0.5, "-" using 1:3 with lines lc rgb "#0020a0" lw 4 title "Traffic ingress (mean %.2f Mbits/s)", "-" using 1:4 with lines lc rgb "#ff6040" lw 2 title "Traffic egress (mean %.2f Mbits/s)"\n},
$first_timestamp / 1000.0, $last_timestamp / 1000.0, $average_capacity, $average_ingress, $average_throughput;
my $output;
for ( my $bin = $earliest_bin; $bin <= $latest_bin; $bin++ ) {
my $t = bin_to_seconds( $bin );
my ( $cap, $arr, $dep ) = map { (default $_) / ($MS_PER_BIN / 1000.0) / 1000000.0 } ( $capacity{ $bin }, $arrivals{ $bin }, $departures{ $bin } );
$current_buffer_occupancy += default $arrivals{ $bin };
$current_buffer_occupancy -= default $departures{ $bin };
$output .= qq{$t $cap $arr $dep $current_buffer_occupancy\n};
}
print GNUPLOT $output;
print GNUPLOT qq{\ne\n};
print GNUPLOT $output;
print GNUPLOT qq{\ne\n};
print GNUPLOT $output;
close GNUPLOT or die qq{$!};
src/Makefile.am
+1
View File
@@ -0,0 +1 @@
SUBDIRS = protobufs util packet graphing http httpserver frontend tests
src/frontend/Makefile.am
+63
View File
@@ -0,0 +1,63 @@
AM_CPPFLAGS = -I../protobufs -I$(srcdir)/../util -I$(srcdir)/../packet -I$(srcdir)/../graphing -I$(srcdir)/../http -I$(srcdir)/../httpserver $(XCBPRESENT_CFLAGS) $(XCB_CFLAGS) $(PANGOCAIRO_CFLAGS) $(CXX11_FLAGS)
AM_CXXFLAGS = $(PICKY_CXXFLAGS)
bin_PROGRAMS = mm-delay
mm_delay_SOURCES = delayshell.cc delay_queue.hh delay_queue.cc
mm_delay_LDADD = -lrt ../util/libutil.a ../packet/libpacket.a
mm_delay_LDFLAGS = -pthread
bin_PROGRAMS += mm-loss
mm_loss_SOURCES = lossshell.cc loss_queue.hh loss_queue.cc
mm_loss_LDADD = -lrt ../util/libutil.a ../packet/libpacket.a
mm_loss_LDFLAGS = -pthread
bin_PROGRAMS += mm-onoff
mm_onoff_SOURCES = onoffshell.cc loss_queue.hh loss_queue.cc
mm_onoff_LDADD = -lrt ../util/libutil.a ../packet/libpacket.a
mm_onoff_LDFLAGS = -pthread
bin_PROGRAMS += mm-link
mm_link_SOURCES = linkshell.cc link_queue.hh link_queue.cc
mm_link_LDADD = -lrt ../util/libutil.a ../packet/libpacket.a ../graphing/libgraph.a $(XCBPRESENT_LIBS) $(XCB_LIBS) $(PANGOCAIRO_LIBS)
mm_link_LDFLAGS = -pthread
bin_PROGRAMS += mm-meter
mm_meter_SOURCES = meter.cc meter_queue.hh meter_queue.cc
mm_meter_LDADD = -lrt ../util/libutil.a ../packet/libpacket.a ../graphing/libgraph.a $(XCBPRESENT_LIBS) $(XCB_LIBS) $(PANGOCAIRO_LIBS)
mm_meter_LDFLAGS = -pthread
bin_PROGRAMS += mm-webrecord
mm_webrecord_SOURCES = recordshell.cc
mm_webrecord_LDADD = -lrt ../httpserver/libhttpserver.a ../http/libhttp.a ../protobufs/libhttprecordprotos.a ../util/libutil.a $(protobuf_LIBS) $(libcrypto_LIBS) $(libssl_LIBS)
mm_webrecord_LDFLAGS = -pthread
bin_PROGRAMS += mm-webreplay
mm_webreplay_SOURCES = replayshell.cc web_server.hh web_server.cc
mm_webreplay_LDADD = -lrt ../util/libutil.a ../http/libhttp.a ../protobufs/libhttprecordprotos.a $(protobuf_LIBS)
mm_webreplay_LDFLAGS = -pthread
bin_PROGRAMS += mm-replayserver
mm_replayserver_SOURCES = replayserver.cc
mm_replayserver_LDADD = -lrt ../util/libutil.a ../http/libhttp.a ../protobufs/libhttprecordprotos.a $(protobuf_LIBS)
mm_replayserver_LDFLAGS = -pthread
lib_LTLIBRARIES = libmod_deepcgi.la
libmod_deepcgi_la_SOURCES = mod_deepcgi.c replayserver_filename.cc
libmod_deepcgi_la_CFLAGS = -I@APACHE2_INCLUDE@ $(libapr1_CFLAGS)
libmod_deepcgi_la_CPPFLAGS = # empty
install-exec-hook:
chown root $(DESTDIR)$(bindir)/mm-delay
chmod u+s $(DESTDIR)$(bindir)/mm-delay
chown root $(DESTDIR)$(bindir)/mm-loss
chmod u+s $(DESTDIR)$(bindir)/mm-loss
chown root $(DESTDIR)$(bindir)/mm-onoff
chmod u+s $(DESTDIR)$(bindir)/mm-onoff
chown root $(DESTDIR)$(bindir)/mm-link
chmod u+s $(DESTDIR)$(bindir)/mm-link
chown root $(DESTDIR)$(bindir)/mm-meter
chmod u+s $(DESTDIR)$(bindir)/mm-meter
chown root $(DESTDIR)$(bindir)/mm-webrecord
chmod u+s $(DESTDIR)$(bindir)/mm-webrecord
chown root $(DESTDIR)$(bindir)/mm-webreplay
chmod u+s $(DESTDIR)$(bindir)/mm-webreplay
src/frontend/delay_queue.cc
+37
View File
@@ -0,0 +1,37 @@
/* -*-mode:c++; tab-width: 4; indent-tabs-mode: nil; c-basic-offset: 4 -*- */
#include <limits>
#include "delay_queue.hh"
#include "timestamp.hh"
using namespace std;
void DelayQueue::read_packet( const string & contents )
{
packet_queue_.emplace( timestamp() + delay_ms_, contents );
}
void DelayQueue::write_packets( FileDescriptor & fd )
{
while ( (!packet_queue_.empty())
&& (packet_queue_.front().first <= timestamp()) ) {
fd.write( packet_queue_.front().second );
packet_queue_.pop();
}
}
unsigned int DelayQueue::wait_time( void ) const
{
if ( packet_queue_.empty() ) {
return numeric_limits<uint16_t>::max();
}
const auto now = timestamp();
if ( packet_queue_.front().first <= now ) {
return 0;
} else {
return packet_queue_.front().first - now;
}
}
src/frontend/delay_queue.hh
+33
View File
@@ -0,0 +1,33 @@
/* -*-mode:c++; tab-width: 4; indent-tabs-mode: nil; c-basic-offset: 4 -*- */
#ifndef DELAY_QUEUE_HH
#define DELAY_QUEUE_HH
#include <queue>
#include <cstdint>
#include <string>
#include "file_descriptor.hh"
class DelayQueue
{
private:
uint64_t delay_ms_;
std::queue< std::pair<uint64_t, std::string> > packet_queue_;
/* release timestamp, contents */
public:
DelayQueue( const uint64_t & s_delay_ms ) : delay_ms_( s_delay_ms ), packet_queue_() {}
void read_packet( const std::string & contents );
void write_packets( FileDescriptor & fd );
unsigned int wait_time( void ) const;
bool pending_output( void ) const { return wait_time() <= 0; }
static bool finished( void ) { return false; }
};
#endif /* DELAY_QUEUE_HH */
src/frontend/delayshell.cc
+49
View File
@@ -0,0 +1,49 @@
/* -*-mode:c++; tab-width: 4; indent-tabs-mode: nil; c-basic-offset: 4 -*- */
#include <vector>
#include <string>
#include "delay_queue.hh"
#include "util.hh"
#include "ezio.hh"
#include "packetshell.cc"
using namespace std;
int main( int argc, char *argv[] )
{
try {
/* clear environment while running as root */
char ** const user_environment = environ;
environ = nullptr;
check_requirements( argc, argv );
if ( argc < 2 ) {
throw runtime_error( "Usage: " + string( argv[ 0 ] ) + " delay-milliseconds [command...]" );
}
const uint64_t delay_ms = myatoi( argv[ 1 ] );
vector< string > command;
if ( argc == 2 ) {
command.push_back( shell_path() );
} else {
for ( int i = 2; i < argc; i++ ) {
command.push_back( argv[ i ] );
}
}
PacketShell<DelayQueue> delay_shell_app( "delay", user_environment );
delay_shell_app.start_uplink( "[delay " + to_string( delay_ms ) + " ms] ",
command,
delay_ms );
delay_shell_app.start_downlink( delay_ms );
return delay_shell_app.wait_for_exit();
} catch ( const exception & e ) {
print_exception( e );
return EXIT_FAILURE;
}
}
src/frontend/link_queue.cc
+279
View File
@@ -0,0 +1,279 @@
/* -*-mode:c++; tab-width: 4; indent-tabs-mode: nil; c-basic-offset: 4 -*- */
#include <limits>
#include <cassert>
#include "link_queue.hh"
#include "timestamp.hh"
#include "util.hh"
#include "ezio.hh"
#include "abstract_packet_queue.hh"
using namespace std;
LinkQueue::LinkQueue( const string & link_name, const string & filename, const string & logfile,
const bool repeat, const bool graph_throughput, const bool graph_delay,
unique_ptr<AbstractPacketQueue> && packet_queue,
const string & command_line )
: next_delivery_( 0 ),
schedule_(),
base_timestamp_( timestamp() ),
packet_queue_( move( packet_queue ) ),
packet_in_transit_( "", 0 ),
packet_in_transit_bytes_left_( 0 ),
output_queue_(),
log_(),
throughput_graph_( nullptr ),
delay_graph_( nullptr ),
repeat_( repeat ),
finished_( false )
{
assert_not_root();
/* open filename and load schedule */
ifstream trace_file( filename );
if ( not trace_file.good() ) {
throw runtime_error( filename + ": error opening for reading" );
}
string line;
while ( trace_file.good() and getline( trace_file, line ) ) {
if ( line.empty() ) {
throw runtime_error( filename + ": invalid empty line" );
}
const uint64_t ms = myatoi( line );
if ( not schedule_.empty() ) {
if ( ms < schedule_.back() ) {
throw runtime_error( filename + ": timestamps must be monotonically nondecreasing" );
}
}
schedule_.emplace_back( ms );
}
if ( schedule_.empty() ) {
throw runtime_error( filename + ": no valid timestamps found" );
}
if ( schedule_.back() == 0 ) {
throw runtime_error( filename + ": trace must last for a nonzero amount of time" );
}
/* open logfile if called for */
if ( not logfile.empty() ) {
log_.reset( new ofstream( logfile ) );
if ( not log_->good() ) {
throw runtime_error( logfile + ": error opening for writing" );
}
*log_ << "# mahimahi mm-link (" << link_name << ") [" << filename << "] > " << logfile << endl;
*log_ << "# command line: " << command_line << endl;
*log_ << "# queue: " << packet_queue_->to_string() << endl;
*log_ << "# init timestamp: " << initial_timestamp() << endl;
*log_ << "# base timestamp: " << base_timestamp_ << endl;
const char * prefix = getenv( "MAHIMAHI_SHELL_PREFIX" );
if ( prefix ) {
*log_ << "# mahimahi config: " << prefix << endl;
}
}
/* create graphs if called for */
if ( graph_throughput ) {
throughput_graph_.reset( new BinnedLiveGraph( link_name + " [" + filename + "]",
{ make_tuple( 1.0, 0.0, 0.0, 0.25, true ),
make_tuple( 0.0, 0.0, 0.4, 1.0, false ),
make_tuple( 1.0, 0.0, 0.0, 0.5, false ) },
"throughput (Mbps)",
8.0 / 1000000.0,
true,
500,
[] ( int, int & x ) { x = 0; } ) );
}
if ( graph_delay ) {
delay_graph_.reset( new BinnedLiveGraph( link_name + " delay [" + filename + "]",
{ make_tuple( 0.0, 0.25, 0.0, 1.0, false ) },
"queueing delay (ms)",
1, false, 250,
[] ( int, int & x ) { x = -1; } ) );
}
}
void LinkQueue::record_arrival( const uint64_t arrival_time, const size_t pkt_size )
{
/* log it */
if ( log_ ) {
*log_ << arrival_time << " + " << pkt_size << endl;
}
/* meter it */
if ( throughput_graph_ ) {
throughput_graph_->add_value_now( 1, pkt_size );
}
}
void LinkQueue::record_drop( const uint64_t time, const size_t pkts_dropped, const size_t bytes_dropped)
{
/* log it */
if ( log_ ) {
*log_ << time << " d " << pkts_dropped << " " << bytes_dropped << endl;
}
}
void LinkQueue::record_departure_opportunity( void )
{
/* log the delivery opportunity */
if ( log_ ) {
*log_ << next_delivery_time() << " # " << PACKET_SIZE << endl;
}
/* meter the delivery opportunity */
if ( throughput_graph_ ) {
throughput_graph_->add_value_now( 0, PACKET_SIZE );
}
}
void LinkQueue::record_departure( const uint64_t departure_time, const QueuedPacket & packet )
{
/* log the delivery */
if ( log_ ) {
*log_ << departure_time << " - " << packet.contents.size()
<< " " << departure_time - packet.arrival_time << endl;
}
/* meter the delivery */
if ( throughput_graph_ ) {
throughput_graph_->add_value_now( 2, packet.contents.size() );
}
if ( delay_graph_ ) {
delay_graph_->set_max_value_now( 0, departure_time - packet.arrival_time );
}
}
void LinkQueue::read_packet( const string & contents )
{
const uint64_t now = timestamp();
if ( contents.size() > PACKET_SIZE ) {
throw runtime_error( "packet size is greater than maximum" );
}
rationalize( now );
record_arrival( now, contents.size() );
unsigned int bytes_before = packet_queue_->size_bytes();
unsigned int packets_before = packet_queue_->size_packets();
packet_queue_->enqueue( QueuedPacket( contents, now ) );
assert( packet_queue_->size_packets() <= packets_before + 1 );
assert( packet_queue_->size_bytes() <= bytes_before + contents.size() );
unsigned int missing_packets = packets_before + 1 - packet_queue_->size_packets();
unsigned int missing_bytes = bytes_before + contents.size() - packet_queue_->size_bytes();
if ( missing_packets > 0 || missing_bytes > 0 ) {
record_drop( now, missing_packets, missing_bytes );
}
}
uint64_t LinkQueue::next_delivery_time( void ) const
{
if ( finished_ ) {
return -1;
} else {
return schedule_.at( next_delivery_ ) + base_timestamp_;
}
}
void LinkQueue::use_a_delivery_opportunity( void )
{
record_departure_opportunity();
next_delivery_ = (next_delivery_ + 1) % schedule_.size();
/* wraparound */
if ( next_delivery_ == 0 ) {
if ( repeat_ ) {
base_timestamp_ += schedule_.back();
} else {
finished_ = true;
}
}
}
/* emulate the link up to the given timestamp */
/* this function should be called before enqueueing any packets and before
calculating the wait_time until the next event */
void LinkQueue::rationalize( const uint64_t now )
{
while ( next_delivery_time() <= now ) {
const uint64_t this_delivery_time = next_delivery_time();
/* burn a delivery opportunity */
unsigned int bytes_left_in_this_delivery = PACKET_SIZE;
use_a_delivery_opportunity();
while ( bytes_left_in_this_delivery > 0 ) {
if ( not packet_in_transit_bytes_left_ ) {
if ( packet_queue_->empty() ) {
break;
}
packet_in_transit_ = packet_queue_->dequeue();
packet_in_transit_bytes_left_ = packet_in_transit_.contents.size();
}
assert( packet_in_transit_.arrival_time <= this_delivery_time );
assert( packet_in_transit_bytes_left_ <= PACKET_SIZE );
assert( packet_in_transit_bytes_left_ > 0 );
assert( packet_in_transit_bytes_left_ <= packet_in_transit_.contents.size() );
/* how many bytes of the delivery opportunity can we use? */
const unsigned int amount_to_send = min( bytes_left_in_this_delivery,
packet_in_transit_bytes_left_ );
/* send that many bytes */
packet_in_transit_bytes_left_ -= amount_to_send;
bytes_left_in_this_delivery -= amount_to_send;
/* has the packet been fully sent? */
if ( packet_in_transit_bytes_left_ == 0 ) {
record_departure( this_delivery_time, packet_in_transit_ );
/* this packet is ready to go */
output_queue_.push( move( packet_in_transit_.contents ) );
}
}
}
}
void LinkQueue::write_packets( FileDescriptor & fd )
{
while ( not output_queue_.empty() ) {
fd.write( output_queue_.front() );
output_queue_.pop();
}
}
unsigned int LinkQueue::wait_time( void )
{
const auto now = timestamp();
rationalize( now );
if ( next_delivery_time() <= now ) {
return 0;
} else {
return next_delivery_time() - now;
}
}
bool LinkQueue::pending_output( void ) const
{
return not output_queue_.empty();
}
src/frontend/link_queue.hh
+66
View File
@@ -0,0 +1,66 @@
/* -*-mode:c++; tab-width: 4; indent-tabs-mode: nil; c-basic-offset: 4 -*- */
#ifndef LINK_QUEUE_HH
#define LINK_QUEUE_HH
#include <queue>
#include <cstdint>
#include <string>
#include <fstream>
#include <memory>
#include "file_descriptor.hh"
#include "binned_livegraph.hh"
#include "abstract_packet_queue.hh"
class LinkQueue
{
private:
const static unsigned int PACKET_SIZE = 1504; /* default max TUN payload size */
unsigned int next_delivery_;
std::vector<uint64_t> schedule_;
uint64_t base_timestamp_;
std::unique_ptr<AbstractPacketQueue> packet_queue_;
QueuedPacket packet_in_transit_;
unsigned int packet_in_transit_bytes_left_;
std::queue<std::string> output_queue_;
std::unique_ptr<std::ofstream> log_;
std::unique_ptr<BinnedLiveGraph> throughput_graph_;
std::unique_ptr<BinnedLiveGraph> delay_graph_;
bool repeat_;
bool finished_;
uint64_t next_delivery_time( void ) const;
void use_a_delivery_opportunity( void );
void record_arrival( const uint64_t arrival_time, const size_t pkt_size );
void record_drop( const uint64_t time, const size_t pkts_dropped, const size_t bytes_dropped );
void record_departure_opportunity( void );
void record_departure( const uint64_t departure_time, const QueuedPacket & packet );
void rationalize( const uint64_t now );
void dequeue_packet( void );
public:
LinkQueue( const std::string & link_name, const std::string & filename, const std::string & logfile,
const bool repeat, const bool graph_throughput, const bool graph_delay,
std::unique_ptr<AbstractPacketQueue> && packet_queue,
const std::string & command_line );
void read_packet( const std::string & contents );
void write_packets( FileDescriptor & fd );
unsigned int wait_time( void );
bool pending_output( void ) const;
bool finished( void ) const { return finished_; }
};
#endif /* LINK_QUEUE_HH */
src/frontend/linkshell.cc
Executable
+198
View File
@@ -0,0 +1,198 @@
/* -*-mode:c++; tab-width: 4; indent-tabs-mode: nil; c-basic-offset: 4 -*- */
#include <getopt.h>
#include "infinite_packet_queue.hh"
#include "drop_tail_packet_queue.hh"
#include "drop_head_packet_queue.hh"
#include "codel_packet_queue.hh"
#include "pie_packet_queue.hh"
#include "link_queue.hh"
#include "packetshell.cc"
using namespace std;
void usage_error( const string & program_name )
{
cerr << "Usage: " << program_name << " UPLINK-TRACE DOWNLINK-TRACE [OPTION]... [COMMAND]" << endl;
cerr << endl;
cerr << "Options = --once" << endl;
cerr << " --uplink-log=FILENAME --downlink-log=FILENAME" << endl;
cerr << " --meter-uplink --meter-uplink-delay" << endl;
cerr << " --meter-downlink --meter-downlink-delay" << endl;
cerr << " --meter-all" << endl;
cerr << " --uplink-queue=QUEUE_TYPE --downlink-queue=QUEUE_TYPE" << endl;
cerr << " --uplink-queue-args=QUEUE_ARGS --downlink-queue-args=QUEUE_ARGS" << endl;
cerr << endl;
cerr << " QUEUE_TYPE = infinite | droptail | drophead | codel | pie" << endl;
cerr << " QUEUE_ARGS = \"NAME=NUMBER[, NAME2=NUMBER2, ...]\"" << endl;
cerr << " (with NAME = bytes | packets | target | interval | qdelay_ref | max_burst)" << endl;
cerr << " target, interval, qdelay_ref, max_burst are in milli-second" << endl << endl;
throw runtime_error( "invalid arguments" );
}
unique_ptr<AbstractPacketQueue> get_packet_queue( const string & type, const string & args, const string & program_name )
{
if ( type == "infinite" ) {
return unique_ptr<AbstractPacketQueue>( new InfinitePacketQueue( args ) );
} else if ( type == "droptail" ) {
return unique_ptr<AbstractPacketQueue>( new DropTailPacketQueue( args ) );
} else if ( type == "drophead" ) {
return unique_ptr<AbstractPacketQueue>( new DropHeadPacketQueue( args ) );
} else if ( type == "codel" ) {
return unique_ptr<AbstractPacketQueue>( new CODELPacketQueue( args ) );
} else if ( type == "pie" ) {
return unique_ptr<AbstractPacketQueue>( new PIEPacketQueue( args ) );
} else {
cerr << "Unknown queue type: " << type << endl;
}
usage_error( program_name );
return nullptr;
}
string shell_quote( const string & arg )
{
string ret = "'";
for ( const auto & ch : arg ) {
if ( ch != '\'' ) {
ret.push_back( ch );
} else {
ret += "'\\''";
}
}
ret += "'";
return ret;
}
int main( int argc, char *argv[] )
{
try {
/* clear environment while running as root */
char ** const user_environment = environ;
environ = nullptr;
check_requirements( argc, argv );
if ( argc < 3 ) {
usage_error( argv[ 0 ] );
}
string command_line { shell_quote( argv[ 0 ] ) }; /* for the log file */
for ( int i = 1; i < argc; i++ ) {
command_line += string( " " ) + shell_quote( argv[ i ] );
}
const option command_line_options[] = {
{ "uplink-log", required_argument, nullptr, 'u' },
{ "downlink-log", required_argument, nullptr, 'd' },
{ "once", no_argument, nullptr, 'o' },
{ "meter-uplink", no_argument, nullptr, 'm' },
{ "meter-downlink", no_argument, nullptr, 'n' },
{ "meter-uplink-delay", no_argument, nullptr, 'x' },
{ "meter-downlink-delay", no_argument, nullptr, 'y' },
{ "meter-all", no_argument, nullptr, 'z' },
{ "uplink-queue", required_argument, nullptr, 'q' },
{ "downlink-queue", required_argument, nullptr, 'w' },
{ "uplink-queue-args", required_argument, nullptr, 'a' },
{ "downlink-queue-args", required_argument, nullptr, 'b' },
{ 0, 0, nullptr, 0 }
};
string uplink_logfile, downlink_logfile;
bool repeat = true;
bool meter_uplink = false, meter_downlink = false;
bool meter_uplink_delay = false, meter_downlink_delay = false;
string uplink_queue_type = "infinite", downlink_queue_type = "infinite",
uplink_queue_args, downlink_queue_args;
while ( true ) {
const int opt = getopt_long( argc, argv, "u:d:", command_line_options, nullptr );
if ( opt == -1 ) { /* end of options */
break;
}
switch ( opt ) {
case 'u':
uplink_logfile = optarg;
break;
case 'd':
downlink_logfile = optarg;
break;
case 'o':
repeat = false;
break;
case 'm':
meter_uplink = true;
break;
case 'n':
meter_downlink = true;
break;
case 'x':
meter_uplink_delay = true;
break;
case 'y':
meter_downlink_delay = true;
break;
case 'z':
meter_uplink = meter_downlink
= meter_uplink_delay = meter_downlink_delay
= true;
break;
case 'q':
uplink_queue_type = optarg;
break;
case 'w':
downlink_queue_type = optarg;
break;
case 'a':
uplink_queue_args = optarg;
break;
case 'b':
downlink_queue_args = optarg;
break;
case '?':
usage_error( argv[ 0 ] );
break;
default:
throw runtime_error( "getopt_long: unexpected return value " + to_string( opt ) );
}
}
if ( optind + 1 >= argc ) {
usage_error( argv[ 0 ] );
}
const string uplink_filename = argv[ optind ];
const string downlink_filename = argv[ optind + 1 ];
vector<string> command;
if ( optind + 2 == argc ) {
command.push_back( shell_path() );
} else {
for ( int i = optind + 2; i < argc; i++ ) {
command.push_back( argv[ i ] );
}
}
PacketShell<LinkQueue> link_shell_app( "link", user_environment );
link_shell_app.start_uplink( "[link] ", command,
"Uplink", uplink_filename, uplink_logfile, repeat, meter_uplink, meter_uplink_delay,
get_packet_queue( uplink_queue_type, uplink_queue_args, argv[ 0 ] ),
command_line );
link_shell_app.start_downlink( "Downlink", downlink_filename, downlink_logfile, repeat, meter_downlink, meter_downlink_delay,
get_packet_queue( downlink_queue_type, downlink_queue_args, argv[ 0 ] ),
command_line );
return link_shell_app.wait_for_exit();
} catch ( const exception & e ) {
print_exception( e );
return EXIT_FAILURE;
}
}
src/frontend/loss_queue.cc
+82
View File
@@ -0,0 +1,82 @@
/* -*-mode:c++; tab-width: 4; indent-tabs-mode: nil; c-basic-offset: 4 -*- */
#include <limits>
#include "loss_queue.hh"
#include "timestamp.hh"
using namespace std;
LossQueue::LossQueue()
: prng_( random_device()() )
{}
void LossQueue::read_packet( const string & contents )
{
if ( not drop_packet( contents ) ) {
packet_queue_.emplace( contents );
}
}
void LossQueue::write_packets( FileDescriptor & fd )
{
while ( not packet_queue_.empty() ) {
fd.write( packet_queue_.front() );
packet_queue_.pop();
}
}
unsigned int LossQueue::wait_time( void )
{
return packet_queue_.empty() ? numeric_limits<uint16_t>::max() : 0;
}
bool IIDLoss::drop_packet( const string & packet __attribute((unused)) )
{
return drop_dist_( prng_ );
}
static const double MS_PER_SECOND = 1000.0;
SwitchingLink::SwitchingLink( const double mean_on_time, const double mean_off_time )
: link_is_on_( false ),
on_process_( 1.0 / (MS_PER_SECOND * mean_off_time) ),
off_process_( 1.0 / (MS_PER_SECOND * mean_on_time) ),
next_switch_time_( timestamp() )
{}
uint64_t bound( const double x )
{
if ( x > (1 << 30) ) {
return 1 << 30;
}
return x;
}
unsigned int SwitchingLink::wait_time( void )
{
const uint64_t now = timestamp();
while ( next_switch_time_ <= now ) {
/* switch */
link_is_on_ = !link_is_on_;
/* worried about integer overflow when mean time = 0 */
next_switch_time_ += bound( (link_is_on_ ? off_process_ : on_process_)( prng_ ) );
}
if ( LossQueue::wait_time() == 0 ) {
return 0;
}
if ( next_switch_time_ - now > numeric_limits<uint16_t>::max() ) {
return numeric_limits<uint16_t>::max();
}
return next_switch_time_ - now;
}
bool SwitchingLink::drop_packet( const string & packet __attribute((unused)) )
{
return !link_is_on_;
}
src/frontend/loss_queue.hh
+68
View File
@@ -0,0 +1,68 @@
/* -*-mode:c++; tab-width: 4; indent-tabs-mode: nil; c-basic-offset: 4 -*- */
#ifndef LOSS_QUEUE_HH
#define LOSS_QUEUE_HH
#include <queue>
#include <cstdint>
#include <string>
#include <random>
#include "file_descriptor.hh"
class LossQueue
{
private:
std::queue<std::string> packet_queue_ {};
virtual bool drop_packet( const std::string & packet ) = 0;
protected:
std::default_random_engine prng_;
public:
LossQueue();
virtual ~LossQueue() {}
void read_packet( const std::string & contents );
void write_packets( FileDescriptor & fd );
unsigned int wait_time( void );
bool pending_output( void ) const { return not packet_queue_.empty(); }
static bool finished( void ) { return false; }
};
class IIDLoss : public LossQueue
{
private:
std::bernoulli_distribution drop_dist_;
bool drop_packet( const std::string & packet ) override;
public:
IIDLoss( const double loss_rate ) : drop_dist_( loss_rate ) {}
};
class SwitchingLink : public LossQueue
{
private:
bool link_is_on_;
std::exponential_distribution<> on_process_;
std::exponential_distribution<> off_process_;
uint64_t next_switch_time_;
void calculate_next_switch_time( void );
bool drop_packet( const std::string & packet ) override;
public:
SwitchingLink( const double mean_on_time_, const double mean_off_time );
unsigned int wait_time( void );
};
#endif /* LOSS_QUEUE_HH */
src/frontend/lossshell.cc
+83
View File
@@ -0,0 +1,83 @@
/* -*-mode:c++; tab-width: 4; indent-tabs-mode: nil; c-basic-offset: 4 -*- */
#include <vector>
#include <string>
#include <iostream>
#include <getopt.h>
#include "loss_queue.hh"
#include "util.hh"
#include "ezio.hh"
#include "packetshell.cc"
using namespace std;
void usage( const string & program_name )
{
throw runtime_error( "Usage: " + program_name + " uplink|downlink RATE [COMMAND...]" );
}
int main( int argc, char *argv[] )
{
try {
/* clear environment while running as root */
char ** const user_environment = environ;
environ = nullptr;
check_requirements( argc, argv );
if ( argc < 3 ) {
usage( argv[ 0 ] );
}
const double loss_rate = myatof( argv[ 2 ] );
if ( (0 <= loss_rate) and (loss_rate <= 1) ) {
/* do nothing */
} else {
cerr << "Error: loss rate must be between 0 and 1." << endl;
usage( argv[ 0 ] );
}
double uplink_loss = 0, downlink_loss = 0;
const string link = argv[ 1 ];
if ( link == "uplink" ) {
uplink_loss = loss_rate;
} else if ( link == "downlink" ) {
downlink_loss = loss_rate;
} else {
usage( argv[ 0 ] );
}
vector<string> command;
if ( argc == 3 ) {
command.push_back( shell_path() );
} else {
for ( int i = 3; i < argc; i++ ) {
command.push_back( argv[ i ] );
}
}
PacketShell<IIDLoss> loss_app( "loss", user_environment );
string shell_prefix = "[loss ";
if ( link == "uplink" ) {
shell_prefix += "up=";
} else {
shell_prefix += "down=";
}
shell_prefix += argv[ 2 ];
shell_prefix += "] ";
loss_app.start_uplink( shell_prefix,
command,
uplink_loss );
loss_app.start_downlink( downlink_loss );
return loss_app.wait_for_exit();
} catch ( const exception & e ) {
print_exception( e );
return EXIT_FAILURE;
}
}
src/frontend/meter.cc
+75
View File
@@ -0,0 +1,75 @@
/* -*-mode:c++; tab-width: 4; indent-tabs-mode: nil; c-basic-offset: 4 -*- */
#include <getopt.h>
#include "meter_queue.hh"
#include "packetshell.cc"
using namespace std;
void usage_error( const string & program_name )
{
throw runtime_error( "Usage: " + program_name + " [--meter-uplink] [--meter-downlink] [COMMAND...]" );
}
int main( int argc, char *argv[] )
{
try {
/* clear environment while running as root */
char ** const user_environment = environ;
environ = nullptr;
check_requirements( argc, argv );
const option command_line_options[] = {
{ "meter-uplink", no_argument, nullptr, 'u' },
{ "meter-downlink", no_argument, nullptr, 'd' },
{ 0, 0, nullptr, 0 }
};
bool meter_uplink = false, meter_downlink = false;
while ( true ) {
const int opt = getopt_long( argc, argv, "ud", command_line_options, nullptr );
if ( opt == -1 ) { /* end of options */
break;
}
switch ( opt ) {
case 'u':
meter_uplink = true;
break;
case 'd':
meter_downlink = true;
break;
case '?':
usage_error( argv[ 0 ] );
break;
default:
throw runtime_error( "getopt_log: unexpected return value " + to_string( opt ) );
}
}
vector< string > command;
if ( optind == argc ) {
command.push_back( shell_path() );
} else {
for ( int i = optind; i < argc; i++ ) {
command.push_back( argv[ i ] );
}
}
PacketShell<MeterQueue> link_shell_app( "meter", user_environment );
const string uplink_name = "Uplink", downlink_name = "Downlink";
link_shell_app.start_uplink( "[meter] ", command,
uplink_name, meter_uplink );
link_shell_app.start_downlink( downlink_name, meter_downlink );
return link_shell_app.wait_for_exit();
} catch ( const exception & e ) {
print_exception( e );
return EXIT_FAILURE;
}
}
src/frontend/meter_queue.cc
+41
View File
@@ -0,0 +1,41 @@
/* -*-mode:c++; tab-width: 4; indent-tabs-mode: nil; c-basic-offset: 4 -*- */
#include "meter_queue.hh"
#include "util.hh"
#include "timestamp.hh"
using namespace std;
MeterQueue::MeterQueue( const string & name, const bool graph )
: packet_queue_(),
graph_( nullptr )
{
assert_not_root();
if ( graph ) {
graph_.reset( new BinnedLiveGraph( name, { make_tuple( 0.0, 0.0, 0.4, 1.0, false ) }, "throughput (Mbps)", 8.0 / 1000000.0, true, 500, [] ( int, int & x ) { x = 0; } ) );
}
}
void MeterQueue::read_packet( const string & contents )
{
packet_queue_.emplace( contents );
/* meter it */
if ( graph_ ) {
graph_->add_value_now( 0, contents.size() );
}
}
void MeterQueue::write_packets( FileDescriptor & fd )
{
while ( not packet_queue_.empty() ) {
fd.write( packet_queue_.front() );
packet_queue_.pop();
}
}
unsigned int MeterQueue::wait_time( void ) const
{
return packet_queue_.empty() ? numeric_limits<uint16_t>::max() : 0;
}
src/frontend/meter_queue.hh
+33
View File
@@ -0,0 +1,33 @@
/* -*-mode:c++; tab-width: 4; indent-tabs-mode: nil; c-basic-offset: 4 -*- */
#ifndef METER_QUEUE_HH
#define METER_QUEUE_HH
#include <queue>
#include <string>
#include <memory>
#include "file_descriptor.hh"
#include "binned_livegraph.hh"
class MeterQueue
{
private:
std::queue<std::string> packet_queue_;
std::unique_ptr<BinnedLiveGraph> graph_;
public:
MeterQueue( const std::string & name, const bool graph );
void read_packet( const std::string & contents );
void write_packets( FileDescriptor & fd );
unsigned int wait_time( void ) const;
bool pending_output( void ) const { return not packet_queue_.empty(); }
static bool finished( void ) { return false; }
};
#endif /* METER_QUEUE_HH */
src/frontend/mod_deepcgi.c
+150
View File
@@ -0,0 +1,150 @@
#include <stdio.h>
#include "httpd.h"
#include "http_core.h"
#include "http_protocol.h"
extern const char* replayserver_filename;
static void deepcgi_hooks( apr_pool_t* inpPool );
static int deepcgi_handler( request_rec* inpRequest );
typedef struct {
const char* working_dir;
const char* recording_dir;
} deepcgi_config;
static deepcgi_config config;
// ============================================================================
// Methods for reading configuration parameters
// ============================================================================
const char* deepcgi_set_workingdir(cmd_parms* cmd, void* cfg, const char* arg) {
config.working_dir = arg;
return NULL;
}
const char* deepcgi_set_recordingdir(cmd_parms* cmd, void* cfg, const char* arg) {
config.recording_dir = arg;
return NULL;
}
// ============================================================================
// Directives to read configuration parameters
// ============================================================================
static const command_rec deepcgi_directives[] =
{
AP_INIT_TAKE1( "workingDir", deepcgi_set_workingdir, NULL, RSRC_CONF, "Working directory" ),
AP_INIT_TAKE1( "recordingDir", deepcgi_set_recordingdir, NULL, RSRC_CONF, "Recording directory" ),
{ NULL }
};
// ============================================================================
// Module definition
// ============================================================================
module AP_MODULE_DECLARE_DATA deepcgi_module =
{
STANDARD20_MODULE_STUFF,
NULL,
NULL,
NULL,
NULL,
deepcgi_directives,
deepcgi_hooks
};
// ============================================================================
// Module handler function
// ============================================================================
int deepcgi_handler( request_rec* inpRequest )
{
if ( !inpRequest->handler || strcmp( inpRequest->handler, "deepcgi-handler" ))
{
return DECLINED;
}
const char* request_method = inpRequest->method;
const char* request_uri = inpRequest->unparsed_uri;
const char* protocol = inpRequest->protocol;
const char* http_host = inpRequest->hostname;
const char* user_agent = apr_table_get( inpRequest->headers_in, "User-Agent" );
const char* range = apr_table_get( inpRequest->headers_in, "Range" );
setenv( "MAHIMAHI_CHDIR", config.working_dir, TRUE );
setenv( "MAHIMAHI_RECORD_PATH", config.recording_dir, TRUE );
setenv( "REQUEST_METHOD", request_method, TRUE );
setenv( "REQUEST_URI", request_uri, TRUE );
setenv( "SERVER_PROTOCOL", protocol, TRUE );
setenv( "HTTP_HOST", http_host, TRUE );
if ( user_agent != NULL ) {
setenv( "HTTP_USER_AGENT", user_agent, TRUE );
}
if ( range != NULL ) {
setenv( "HTTP_RANGE", range, TRUE );
}
/* check if connection is HTTPS */
/* see bug report at http://modpython.org/pipermail/mod_python/2006-February/020197.html */
/* taken from fix at https://issues.apache.org/jira/secure/attachment/12321011/requestobject.c.patch */
APR_DECLARE_OPTIONAL_FN(int, ssl_is_https, (conn_rec *));
APR_OPTIONAL_FN_TYPE(ssl_is_https) *optfn_is_https
= APR_RETRIEVE_OPTIONAL_FN(ssl_is_https);
if ( optfn_is_https ) {
if ( optfn_is_https( inpRequest->connection ) ) {
setenv( "HTTPS", "1", TRUE );
}
}
FILE* fp = popen( replayserver_filename, "r" );
if ( fp == NULL ) {
// "Error encountered while running script"
return HTTP_INTERNAL_SERVER_ERROR;
}
char line[HUGE_STRING_LEN];
struct ap_filter_t *cur;
// Get rid of all filters up through protocol...since we
// haven't parsed off the headers, there is no way they can
// work
cur = inpRequest->proto_output_filters;
while (cur && cur->frec->ftype < AP_FTYPE_CONNECTION) {
cur = cur->next;
}
inpRequest->output_filters = inpRequest->proto_output_filters = cur;
// Write headers + body
int num_bytes_read;
do {
num_bytes_read = fread( line, sizeof(char), HUGE_STRING_LEN, fp );
int num_bytes_left = num_bytes_read;
while ( num_bytes_left > 0 ) {
int offset = num_bytes_read - num_bytes_left;
int num_bytes_written = ap_rwrite( line + offset, num_bytes_left, inpRequest );
if ( num_bytes_written == -1 ) {
// "Error encountered while writing"
return HTTP_INTERNAL_SERVER_ERROR;
}
num_bytes_left -= num_bytes_written;
}
} while ( num_bytes_read == HUGE_STRING_LEN );
// To ensure that connection is kept-alive
ap_set_keepalive( inpRequest );
pclose( fp );
return OK;
}
// ============================================================================
// Definition of hook for handler
// ============================================================================
void deepcgi_hooks( apr_pool_t* inpPool )
{
ap_hook_handler( deepcgi_handler, NULL, NULL, APR_HOOK_LAST );
}
src/frontend/onoffshell.cc
+101
View File
@@ -0,0 +1,101 @@
/* -*-mode:c++; tab-width: 4; indent-tabs-mode: nil; c-basic-offset: 4 -*- */
#include <vector>
#include <string>
#include <iostream>
#include <getopt.h>
#include "loss_queue.hh"
#include "util.hh"
#include "ezio.hh"
#include "packetshell.cc"
using namespace std;
void usage( const string & program_name )
{
throw runtime_error( "Usage: " + program_name + " uplink|downlink MEAN-ON-TIME MEAN-OFF-TIME [COMMAND...]" );
}
int main( int argc, char *argv[] )
{
try {
/* clear environment while running as root */
char ** const user_environment = environ;
environ = nullptr;
check_requirements( argc, argv );
if ( argc < 4 ) {
usage( argv[ 0 ] );
}
const double on_time = myatof( argv[ 2 ] );
if ( (0 <= on_time) ) {
/* do nothing */
} else {
cerr << "Error: mean on-time must be more than 0 seconds." << endl;
usage( argv[ 0 ] );
}
const double off_time = myatof( argv[ 3 ] );
if ( (0 <= off_time) ) {
/* do nothing */
} else {
cerr << "Error: mean off-time must be more than 0 seconds." << endl;
usage( argv[ 0 ] );
}
if ( on_time == 0 and off_time == 0 ) {
cerr << "Error: mean on-time and off-time cannot both be 0 seconds." << endl;
usage( argv[ 0 ] );
}
double uplink_on_time = numeric_limits<double>::max(), uplink_off_time = 0;
double downlink_on_time = numeric_limits<double>::max(), downlink_off_time = 0;
const string link = argv[ 1 ];
if ( link == "uplink" ) {
uplink_on_time = on_time;
uplink_off_time = off_time;
} else if ( link == "downlink" ) {
downlink_on_time = on_time;
downlink_off_time = off_time;
} else {
usage( argv[ 0 ] );
}
vector<string> command;
if ( argc == 4 ) {
command.push_back( shell_path() );
} else {
for ( int i = 4; i < argc; i++ ) {
command.push_back( argv[ i ] );
}
}
PacketShell<SwitchingLink> onoff_app( "onoff", user_environment );
string shell_prefix = "[onoff ";
if ( link == "uplink" ) {
shell_prefix += "(up) on=";
} else {
shell_prefix += "(down) on=";
}
shell_prefix += argv[ 2 ];
shell_prefix += "s off=";
shell_prefix += argv[ 3 ];
shell_prefix += "s] ";
onoff_app.start_uplink( shell_prefix,
command,
uplink_on_time, uplink_off_time );
onoff_app.start_downlink( downlink_on_time, downlink_off_time );
return onoff_app.wait_for_exit();
} catch ( const exception & e ) {
print_exception( e );
return EXIT_FAILURE;
}
}
src/frontend/recordshell.cc
+168
View File
@@ -0,0 +1,168 @@
/* -*-mode:c++; tab-width: 4; indent-tabs-mode: nil; c-basic-offset: 4 -*- */
#include <sys/socket.h>
#include <sys/ioctl.h>
#include <linux/if.h>
#include <net/route.h>
#include "nat.hh"
#include "util.hh"
#include "interfaces.hh"
#include "address.hh"
#include "dns_proxy.hh"
#include "http_proxy.hh"
#include "netdevice.hh"
#include "event_loop.hh"
#include "socketpair.hh"
#include "config.h"
#include "backing_store.hh"
#include "exception.hh"
using namespace std;
int main( int argc, char *argv[] )
{
try {
/* clear environment */
char **user_environment = environ;
environ = nullptr;
check_requirements( argc, argv );
if ( argc < 2 ) {
throw runtime_error( "Usage: " + string( argv[ 0 ] ) + " directory [command...]" );
}
/* Make sure directory ends with '/' so we can prepend directory to file name for storage */
string directory( argv[ 1 ] );
if ( directory.empty() ) {
throw runtime_error( string( argv[ 0 ] ) + ": directory name must be non-empty" );
}
if ( directory.back() != '/' ) {
directory.append( "/" );
}
/* what command will we run inside the container? */
vector < string > command;
if ( argc == 2 ) {
command.push_back( shell_path() );
} else {
for ( int i = 2; i < argc; i++ ) {
command.push_back( argv[ i ] );
}
}
const Address nameserver = first_nameserver();
/* set egress and ingress ip addresses */
Address egress_addr, ingress_addr;
tie( egress_addr, ingress_addr ) = two_unassigned_addresses();
/* make pair of devices */
string egress_name = "veth-" + to_string( getpid() ), ingress_name = "veth-i" + to_string( getpid() );
VirtualEthernetPair veth_devices( egress_name, ingress_name );
/* bring up egress */
assign_address( egress_name, egress_addr, ingress_addr );
/* create DNS proxy */
DNSProxy dns_outside( egress_addr, nameserver, nameserver );
/* set up NAT between egress and eth0 */
NAT nat_rule( ingress_addr );
/* set up http proxy for tcp */
HTTPProxy http_proxy( egress_addr );
/* set up dnat */
DNAT dnat( http_proxy.tcp_listener().local_address(), egress_name );
/* prepare event loop */
EventLoop outer_event_loop;
/* Fork */
{
/* Make pipe for start signal */
auto pipe = UnixDomainSocket::make_pair();
ChildProcess container_process( "recordshell", [&]() {
/* wait for the go signal */
pipe.second.read();
/* bring up localhost */
interface_ioctl( SIOCSIFFLAGS, "lo",
[] ( ifreq &ifr ) { ifr.ifr_flags = IFF_UP; } );
/* bring up veth device */
assign_address( ingress_name, ingress_addr, egress_addr );
/* create default route */
rtentry route;
zero( route );
route.rt_gateway = egress_addr.to_sockaddr();
route.rt_dst = route.rt_genmask = Address().to_sockaddr();
route.rt_flags = RTF_UP | RTF_GATEWAY;
SystemCall( "ioctl SIOCADDRT", ioctl( UDPSocket().fd_num(), SIOCADDRT, &route ) );
/* create DNS proxy if nameserver address is local */
auto dns_inside = DNSProxy::maybe_proxy( nameserver,
dns_outside.udp_listener().local_address(),
dns_outside.tcp_listener().local_address() );
/* Fork again after dropping root privileges */
drop_privileges();
/* prepare child's event loop */
EventLoop shell_event_loop;
shell_event_loop.add_child_process( join( command ), [&]() {
/* restore environment and tweak prompt */
environ = user_environment;
prepend_shell_prefix( "[record] " );
return ezexec( command, true );
} );
if ( dns_inside ) {
dns_inside->register_handlers( shell_event_loop );
}
return shell_event_loop.loop();
}, true ); /* new network namespace */
/* give ingress to container */
run( { IP, "link", "set", "dev", ingress_name, "netns", to_string( container_process.pid() ) } );
veth_devices.set_kernel_will_destroy();
/* tell ChildProcess it's ok to proceed */
pipe.first.write( "x" );
/* now that we have its pid, move container process to event loop */
outer_event_loop.add_child_process( move( container_process ) );
}
/* do the actual recording in a different unprivileged child */
outer_event_loop.add_child_process( "recorder", [&]() {
drop_privileges();
make_directory( directory );
/* set up backing store to save to disk */
HTTPDiskStore disk_backing_store( directory );
EventLoop recordr_event_loop;
dns_outside.register_handlers( recordr_event_loop );
http_proxy.register_handlers( recordr_event_loop, disk_backing_store );
return recordr_event_loop.loop();
} );
return outer_event_loop.loop();
} catch ( const exception & e ) {
print_exception( e );
return EXIT_FAILURE;
}
}
src/frontend/replayserver.cc
+202
View File
@@ -0,0 +1,202 @@
/* -*-mode:c++; tab-width: 4; indent-tabs-mode: nil; c-basic-offset: 4 -*- */
#include <sys/types.h>
#include <sys/stat.h>
#include <fcntl.h>
#include <unistd.h>
#include <fstream>
#include <iostream>
#include <vector>
#include <limits>
#include "util.hh"
#include "http_record.pb.h"
#include "http_header.hh"
#include "exception.hh"
#include "http_request.hh"
#include "http_response.hh"
#include "file_descriptor.hh"
using namespace std;
string safe_getenv( const string & key )
{
const char * const value = getenv( key.c_str() );
if ( not value ) {
throw runtime_error( "missing environment variable: " + key );
}
return value;
}
/* does the actual HTTP header match this stored request? */
bool header_match( const string & env_var_name,
const string & header_name,
const HTTPRequest & saved_request )
{
const char * const env_value = getenv( env_var_name.c_str() );
/* case 1: neither header exists (OK) */
if ( (not env_value) and (not saved_request.has_header( header_name )) ) {
return true;
}
/* case 2: headers both exist (OK if values match) */
if ( env_value and saved_request.has_header( header_name ) ) {
return saved_request.get_header_value( header_name).find(env_value) != string::npos;
}
/* case 3: one exists but the other doesn't (failure) */
return false;
}
/* compare request_line and certain headers of incoming request and stored request */
unsigned int match_score( const MahimahiProtobufs::RequestResponse & saved_record,
const string & request_line,
const bool is_https,
std::string &error )
{
const HTTPRequest saved_request( saved_record.request() );
bool options = false;
if (request_line.find("OPTIONS") != string::npos) {
options = true;
}
/* match HTTP/HTTPS */
if ( is_https and (saved_record.scheme() != MahimahiProtobufs::RequestResponse_Scheme_HTTPS) ) {
error = "schema not matched";
return 0;
}
if ( (not is_https) and (saved_record.scheme() != MahimahiProtobufs::RequestResponse_Scheme_HTTP) ) {
error = "schema not matched";
return 0;
}
/* match host header */
if ( saved_request.has_header("Host") ) {
if ( not header_match( "HTTP_HOST", "Host", saved_request ) ) {
error = "host not matched";
return 0;
}
}
else if ( saved_request.has_header(":authority"))
{
const char * const env_value = getenv( "HTTP_HOST" );
if (saved_request.get_header_value( ":authority" ) != string( env_value ))
{
error = std::string("authority not matched ") + env_value + " <-> " +saved_request.get_header_value( ":authority" );
return 0;
}
}
else
{
error = "authority, origin or host not set";
return 0;
}
/* match range (ignore range if method is OPTIONS) */
if (!options) {
if (saved_request.has_header("Range")) {
if ( not header_match( "HTTP_RANGE", "Range", saved_request ) ) {
error = "range_mismatch";
return 0;
}
}
}
/* must match first line up to "?" at least */
if ( strip_query( request_line ) != strip_query( saved_request.first_line() ) ) {
error = "prefix_mismatch";
return 0;
}
/* success! return size of common prefix */
const auto max_match = min( request_line.size(), saved_request.first_line().size() );
for ( unsigned int i = 0; i < max_match; i++ ) {
if ( request_line.at( i ) != saved_request.first_line().at( i ) ) {
return i;
}
}
return max_match;
}
int main( void )
{
try {
assert_not_root();
const string working_directory = safe_getenv( "MAHIMAHI_CHDIR" );
const string recording_directory = safe_getenv( "MAHIMAHI_RECORD_PATH" );
const string request_line = safe_getenv( "REQUEST_METHOD" )
+ " " + safe_getenv( "REQUEST_URI" )
+ " " + safe_getenv( "SERVER_PROTOCOL" );
const bool is_https = getenv( "HTTPS" );
SystemCall( "chdir", chdir( working_directory.c_str() ) );
const vector< string > files = list_directory_contents( recording_directory );
unsigned int best_score = 0;
std::vector<MahimahiProtobufs::RequestResponse> best_matches;
auto stripped = strip_query(request_line);
string hash = to_string(hash32(reinterpret_cast<const uint8_t*>(stripped.c_str()),stripped.size()));
int num_files = 0;
std::vector<std::string> errors;
for ( const auto & filename : files ) {
shared_ptr<char> ts1(strdup(filename.c_str()));
string basefilename = string(basename(ts1.get()));
auto cutted = basefilename.substr(0,min(hash.size(),basefilename.size()));
if(cutted != hash)
{
continue;
}
num_files++;
FileDescriptor fd( SystemCall( "open", open( filename.c_str(), O_RDONLY ) ) );
MahimahiProtobufs::RequestResponse current_record;
if ( not current_record.ParseFromFileDescriptor( fd.fd_num() ) ) {
throw runtime_error( filename + ": invalid HTTP request/response" );
}
std::string error;
unsigned int score = match_score( current_record, request_line, is_https, error);
errors.push_back(error);
if ( score > best_score ) {
best_matches.clear();
best_matches.push_back(current_record);
best_score = score;
}
else if (score == best_score)
{
best_matches.push_back(current_record);
}
}
if ( best_score > 0 ) { /* give client the best match */
cout << HTTPResponse( best_matches[0].response() ).str();
return EXIT_SUCCESS;
} else { /* no acceptable matches for request */
cout << "HTTP/1.1 404 Not Found" << CRLF;
cout << "Content-Type: text/plain" << CRLF << CRLF;
cout << "Hash:" << hash << CRLF;
for(auto error : errors)
{
cout << "Error:" << error << CRLF;
}
cout << "replayserver: could not find a match for " << request_line << CRLF;
cout << "Files considered: " << num_files << CRLF;
return EXIT_FAILURE;
}
} catch ( const exception & e ) {
cout << "HTTP/1.1 500 Internal Server Error" << CRLF;
cout << "Content-Type: text/plain" << CRLF << CRLF;
cout << "mahimahi mm-webreplay received an exception:" << CRLF << CRLF;
print_exception( e, cout );
return EXIT_FAILURE;
}
}
src/frontend/replayserver_filename.cc
+3
View File
@@ -0,0 +1,3 @@
#include "config.h"
const char *replayserver_filename = REPLAYSERVER;
src/frontend/replayshell.cc
+193
View File
@@ -0,0 +1,193 @@
/* -*-mode:c++; tab-width: 4; indent-tabs-mode: nil; c-basic-offset: 4 -*- */
#include <net/route.h>
#include <fcntl.h>
#include <vector>
#include <set>
#include "util.hh"
#include "netdevice.hh"
#include "web_server.hh"
#include "system_runner.hh"
#include "socket.hh"
#include "event_loop.hh"
#include "temp_file.hh"
#include "http_response.hh"
#include "dns_server.hh"
#include "exception.hh"
#include "server_certificate.hh"
#include "nginx_configuration.hh"
#include "http_record.pb.h"
#include "config.h"
using namespace std;
void add_dummy_interface( const string & name, const Address & addr )
{
run( { IP, "link", "add", name, "type", "dummy" } );
interface_ioctl( SIOCSIFFLAGS, name,
[] ( ifreq &ifr ) { ifr.ifr_flags = IFF_UP; } );
interface_ioctl( SIOCSIFADDR, name,
[&] ( ifreq &ifr ) { ifr.ifr_addr = addr.to_sockaddr(); } );
}
int main( int argc, char *argv[] )
{
try {
/* clear environment */
char **user_environment = environ;
environ = nullptr;
check_requirements( argc, argv );
if ( argc < 2 ) {
throw runtime_error( "Usage: " + string( argv[ 0 ] ) + " directory [command...]" );
}
/* clean directory name */
string directory = argv[ 1 ];
if ( directory.empty() ) {
throw runtime_error( string( argv[ 0 ] ) + ": directory name must be non-empty" );
}
/* make sure directory ends with '/' so we can prepend directory to file name for storage */
if ( directory.back() != '/' ) {
directory.append( "/" );
}
/* get working directory */
const string working_directory { get_working_directory() };
/* chdir to result of getcwd just in case */
SystemCall( "chdir", chdir( working_directory.c_str() ) );
/* what command will we run inside the container? */
vector< string > command;
if ( argc == 2 ) {
command.push_back( shell_path() );
} else {
for ( int i = 2; i < argc; i++ ) {
command.push_back( argv[ i ] );
}
}
/* create a new network namespace */
SystemCall( "unshare", unshare( CLONE_NEWNET ) );
/* bring up localhost */
interface_ioctl( SIOCSIFFLAGS, "lo",
[] ( ifreq &ifr ) { ifr.ifr_flags = IFF_UP; } );
/* provide seed for random number generator used to create apache pid files */
srandom( time( NULL ) );
/* collect the IPs, IPs and ports, and hostnames we'll need to serve */
set< Address > unique_ip;
set< Address > unique_ip_and_port;
vector< pair< string, Address > > hostname_to_ip;
{
TemporarilyUnprivileged tu;
/* would be privilege escalation if we let the user read directories or open files as root */
const vector< string > files = list_directory_contents( directory );
for ( const auto filename : files ) {
FileDescriptor fd( SystemCall( "open", open( filename.c_str(), O_RDONLY ) ) );
MahimahiProtobufs::RequestResponse protobuf;
if ( not protobuf.ParseFromFileDescriptor( fd.fd_num() ) ) {
throw runtime_error( filename + ": invalid HTTP request/response" );
}
const Address address( protobuf.ip(), protobuf.port() );
unique_ip.emplace( address.ip(), 0 );
unique_ip_and_port.emplace( address );
int pos = 0;
std::string hostname;
hostname = HTTPRequest( protobuf.request() ).get_header_value( "Host" );
pos = hostname.find(":");
hostname_to_ip.emplace_back( hostname.substr(0, pos), address );
}
}
/* set up dummy interfaces */
unsigned int interface_counter = 0;
for ( const auto ip : unique_ip ) {
add_dummy_interface( "sharded" + to_string( interface_counter ), ip );
interface_counter++;
}
std::cout << "mahimahi replay working_directory : " <<working_directory << std::endl;
CAEnvironment caenv("/data");
/* set up web servers */
vector< std::unique_ptr<ServerCertificate> > certificates;
vector< WebServer > servers;
std::set<std::string> servernames;
string server_name;
char* ca_key;
ca_key = getpass ("enter priavet_key password : ");
for ( const auto ip_port : unique_ip_and_port ) {
servernames.clear();
for ( const auto mapping : hostname_to_ip ) {
if (mapping.second== ip_port) {
servernames.insert(mapping.first);
}
}
/* generate self-signed certificates */
std::unique_ptr<ServerCertificate> p1(new ServerCertificate(servernames, caenv, ca_key));
std::string keyfile = p1->privatekey_file->name();
std::string certfile = p1->certificate_file->name();
certificates.emplace_back(std::move(p1));
servers.emplace_back( ip_port, servernames, working_directory, directory, keyfile, certfile);
}
/* set up DNS server */
TempFile dnsmasq_hosts( "/tmp/replayshell_hosts" );
for ( const auto mapping : hostname_to_ip ) {
std::cout << "dns mapping: " << mapping.second.ip() << " " << mapping.first << std::endl;
dnsmasq_hosts.write( mapping.second.ip() + " " + mapping.first + "\n" );
}
/* initialize event loop */
EventLoop event_loop;
/* create dummy interface for each nameserver */
vector< Address > nameservers = all_nameservers();
vector< string > dnsmasq_args = { "-H", dnsmasq_hosts.name() };
// DNS service
for ( unsigned int server_num = 0; server_num < nameservers.size(); server_num++ ) {
const string interface_name = "nameserver" + to_string( server_num );
std::cout << "add_dummy_interface: interface name: " << interface_name << ", " << nameservers.at( server_num ).str() << std::endl;
add_dummy_interface( interface_name, nameservers.at( server_num ) );
}
/* start dnsmasq */
event_loop.add_child_process( start_dnsmasq( dnsmasq_args ) );
/* start shell */
event_loop.add_child_process( join( command ), [&]() {
drop_privileges();
/* restore environment and tweak bash prompt */
environ = user_environment;
prepend_shell_prefix( "[replay] " );
return ezexec( command, true );
} );
return event_loop.loop();
} catch ( const exception & e ) {
print_exception( e );
return EXIT_FAILURE;
}
}
src/frontend/web_server.cc
+158
View File
@@ -0,0 +1,158 @@
/* -*-mode:c++; tab-width: 4; indent-tabs-mode: nil; c-basic-offset: 4 -*- */
#include <string>
#include <iostream>
#include <fstream>
#include <sstream>
#include <iterator>
#include <unistd.h>
#include <sys/types.h>
#include <sys/stat.h>
#include "web_server.hh"
#include "nginx_configuration.hh"
#include "system_runner.hh"
#include "config.h"
#include "util.hh"
#include "exception.hh"
using namespace std;
WebServer::WebServer( const Address & addr, const std::set<std::string> & servernames, const string & working_directory, const string & record_path, const std::string & keyfile, const std::string & certfile )
: config_file_( "/tmp/replayshell_nginx_config" ),
nginx_pid_file_("/tmp/replayshell_nginx_pidfile"),
fastcgi_pid_file_("/tmp/replayshell_fastcgi_pidfile." + to_string(getpid())),
nginxsrv(nullptr),
moved_away_( false )
{
nginx_pid_file_ = "/tmp/replayshell_nginx_pid." + to_string( getpid() ) + "." + to_string( random() );
std::ostringstream stream;
std::copy(servernames.begin(), servernames.end(), std::ostream_iterator<std::string>(stream, " "));
std::string server_names = stream.str();
if (server_names.length() > 0) {
server_names.substr(0, server_names.length()-1);
}
std::cout << "certfile : " << certfile << std::endl;
std::cout << "keyfile : " << keyfile << std::endl;
config_file_.write( nginx_main_cfg );
config_file_.write( "pid "+nginx_pid_file_+ ";\n" );
config_file_.write( "quic {\n" );
config_file_.write( " quic_stack stack1 {\n" );
config_file_.write( " quic_listen "+addr.str()+";\n");
config_file_.write( " quic_max_streams_per_connection 88;\n");
config_file_.write( " quic_initial_idle_timeout_in_sec 5;\n");
config_file_.write( " quic_default_idle_timeout_in_sec 10;\n");
config_file_.write( " quic_max_idle_timeout_in_sec 10;\n");
config_file_.write( " quic_max_time_before_crypto_handshake_in_sec 10;\n");
config_file_.write( " quic_session_buffer_size 1M;\n");
config_file_.write( " quic_max_age 600;\n");
config_file_.write( " quic_parameter_server http://127.0.0.1;\n");
config_file_.write( " }\n" );
config_file_.write( "}\n" );
config_file_.write( "http {\n" );
config_file_.write( " include /data/tools/nginx/conf/mime.types;\n" );
config_file_.write( " default_type application/octet-stream;\n" );
config_file_.write( " log_format main '$remote_addr - $remote_user [$time_local] \"$request\" '\n" );
config_file_.write( " '$status $body_bytes_sent \"$http_referer\" '\n" );
config_file_.write( " '\"$http_user_agent\" \"$http_x_forwarded_for\"';\n" );
config_file_.write( " access_log logs/access.log main;\n" );
config_file_.write( " error_log logs/error_http.log debug;\n" );
config_file_.write( " sendfile on;\n" );
config_file_.write( " #tcp_nopush on;\n" );
config_file_.write( " keepalive_timeout 65;\n" );
config_file_.write( " #gzip on;\n" );
config_file_.write( " server {\n" );
if ( addr.port() != 80 ) {
std::cout << "WebServer init ssl, ip: " << addr.str() << std::endl;
config_file_.write( " listen "+addr.str()+" ssl http2;\n");
config_file_.write( " server_name "+server_names+";\n");
config_file_.write( " ssl_certificate "+certfile+";\n");
config_file_.write( " ssl_certificate_key "+keyfile+";\n");
config_file_.write( " ssl_session_timeout 30m;\n");
config_file_.write( " ssl_protocols TLSv1.3 TLSv1.2 TLSv1.1 TLSv1;\n");
config_file_.write( " ssl_ciphers EECDH+CHACHA20:EECDH+CHACHA20-draft:EECDH+ECDSA+AES128:RSA+AES128:EECDH+ECDSA+AES256:RSA+AES256:EECDH+3DES:RSA+3DES:!MD5;\n");
config_file_.write( " ssl_session_cache shared:SSL:50m;\n");
config_file_.write( " ssl_prefer_server_ciphers on;\n");
// quic configuration
config_file_.write( " enable_quic stack1;\n");
} else {
std::cout << "NOT WebServer init ssl, ip: " << addr.str() << std::endl;
config_file_.write( " listen "+addr.str()+";\n");
config_file_.write( " server_name "+server_names+";\n");
}
config_file_.write( " location / {\n" );
config_file_.write( " fastcgi_pass " +addr.ip()+ ":9001;\n" );
config_file_.write( " include /data/tools/nginx/conf/fastcgi.conf;\n" );
config_file_.write( " fastcgi_param SCRIPT_FILENAME /data/mahimahi/fcgi/FcgiHandler.py;\n" );
config_file_.write( " fastcgi_param NGINX_CUSTOM_SCHEDULER 0;\n" );
config_file_.write( " fastcgi_param PUSH_STRATEGY_FILE noop;\n" );
config_file_.write( " fastcgi_param WORKING_DIR \""+working_directory+"\";\n" );
config_file_.write( " fastcgi_param RECORDING_DIR \""+record_path+"\";\n" );
config_file_.write( " fastcgi_param REPLAYSERVER_FN \""+string(REPLAYSERVER)+"\";\n" );
config_file_.write( " add_header alt-svc 'h3-29=\":443\"; ma=900, h3-Q050=\":443\"; ma=900,h3-27=\":443\"; ma=900,h3-T051=\":443\"; ma=900,h3-T050=\":443\"; ma=900,h3-Q046=\":443\"; ma=900,h3-Q043=\":443\"; ma=900, quic=\":443\"; ma=900; v=\"46,43\"';\n" );
config_file_.write( " }\n" );
// end of server block
config_file_.write( " }\n" );
// end of http block
config_file_.write( "}\n" );
vector< string > args = { "/data/tools/nginx/sbin/nginx", "-c", config_file_.name() };
run(args);
/* set fastcgi environment */
SystemCall( "setenv", setenv( "NGINX_CUSTOM_SCHEDULER", "0", false) );
SystemCall( "setenv", setenv( "PUSH_STRATEGY_FILE", "noop", false) );
chmod(fastcgi_pid_file_.name().data(), 0666);
vector<string> fcgi_args = {"/usr/local/bin/spawn-fcgi",
"-a", addr.ip(),
"-p", "9001",
"-P", fastcgi_pid_file_.name(),
"-f", "/data/mahimahi/fcgi/FcgiHandler.py"};
int status = system( join(fcgi_args).data() );
if(status == -1) {
std::cout << "system error" << std::endl;
}
environ = nullptr;
}
WebServer::~WebServer()
{
if ( moved_away_ ) { return; }
try {
// kill nginx thread
vector< string > args = { "/data/tools/nginx/sbin/nginx", "-c", config_file_.name(), "-s", "stop"};
run(args);
// kill fcgi thread
std::string pid = fastcgi_pid_file_.fd().read();
if (!pid.empty())
kill(std::stoi(pid), SIGTERM);
return;
} catch ( const exception & e ) { /* don't throw from destructor */
print_exception( e );
}
}
WebServer::WebServer( WebServer && other )
: config_file_( move( other.config_file_ ) ),
nginx_pid_file_( move( other.nginx_pid_file_ ) ),
fastcgi_pid_file_( move( other.fastcgi_pid_file_ ) ),
nginxsrv(move(other.nginxsrv)),
moved_away_( false )
{
other.nginxsrv = nullptr;
other.moved_away_ = true;
}
src/frontend/web_server.hh
+39
View File
@@ -0,0 +1,39 @@
/* -*-mode:c++; tab-width: 4; indent-tabs-mode: nil; c-basic-offset: 4 -*- */
#ifndef WEB_SERVER_HH
#define WEB_SERVER_HH
#include <string>
#include <set>
#include "temp_file.hh"
#include "address.hh"
#include "child_process.hh"
class WebServer
{
private:
/* each apache instance needs unique configuration file, error/access logs, and pid file */
TempFile config_file_;
std::string nginx_pid_file_;
TempFile fastcgi_pid_file_;
ChildProcess* nginxsrv;
bool moved_away_;
public:
WebServer( const Address & addr, const std::set<std::string> & servernames, const std::string & working_directory, const std::string & record_path, const std::string & keyfile, const std::string & certfile);
~WebServer();
/* ban copying */
WebServer( const WebServer & other ) = delete;
WebServer & operator=( const WebServer & other ) = delete;
/* allow move constructor */
WebServer( WebServer && other );
/* ... but not move assignment operator */
WebServer & operator=( WebServer && other ) = delete;
};
#endif
src/graphing/Makefile.am
+9
View File
@@ -0,0 +1,9 @@
AM_CPPFLAGS = -I$(srcdir)/../util $(XCBPRESENT_CFLAGS) $(XCB_CFLAGS) $(PANGOCAIRO_CFLAGS) $(CXX11_FLAGS)
AM_CXXFLAGS = $(PICKY_CXXFLAGS)
noinst_LIBRARIES = libgraph.a
libgraph_a_SOURCES = cairo_objects.hh cairo_objects.cc \
display.hh display.cc \
graph.hh graph.cc \
binned_livegraph.hh binned_livegraph.cc
src/graphing/binned_livegraph.cc
+136
View File
@@ -0,0 +1,136 @@
/* -*-mode:c++; tab-width: 4; indent-tabs-mode: nil; c-basic-offset: 4 -*- */
#include <cmath>
#include <cassert>
#include "binned_livegraph.hh"
#include "timestamp.hh"
#include "exception.hh"
using namespace std;
BinnedLiveGraph::BinnedLiveGraph( const string & name,
const Graph::StylesType & styles,
const string & y_label,
const double multiplier,
const bool rate_quantity,
const unsigned int bin_width_ms,
const function<void(int,int&)> initialize_new_bin )
: graph_( 640, 480, name, 0, 1, styles, "time (s)", y_label ),
bin_width_ms_( bin_width_ms ),
value_this_bin_( styles.size() ),
current_bin_( timestamp() / bin_width_ms_ ),
multiplier_( multiplier ),
rate_quantity_( rate_quantity ),
mutex_(),
halt_( false ),
animation_thread_exception_(),
animation_thread_( [&] () {
try {
animation_loop();
} catch ( ... ) {
animation_thread_exception_ = current_exception();
} } ),
initialize_new_bin_( initialize_new_bin )
{
for ( unsigned int i = 0; i < value_this_bin_.size(); i++ ) {
graph_.add_data_point( i, 0, 0 );
}
}
double BinnedLiveGraph::logical_width( void ) const
{
return max( 5.0, graph_.size().first / 100.0 );
}
void BinnedLiveGraph::animation_loop( void )
{
while ( not halt_ ) {
const uint64_t ts = advance();
/* calculate "current" estimate based on partial bin */
const double bin_width_so_far = ts % bin_width_ms_;
vector<float> current_estimates;
current_estimates.reserve( value_this_bin_.size() );
for ( const auto & x : value_this_bin_ ) {
double current_estimate = x * multiplier_;
if ( rate_quantity_ ) {
current_estimate /= (bin_width_so_far / 1000.0);
}
current_estimates.emplace_back( current_estimate );
}
const double bin_fraction = bin_width_so_far / double( bin_width_ms_ );
const double confidence = pow( 1 - cos( bin_fraction * 3.14159 / 2.0 ), 2 );
graph_.blocking_draw( ts / 1000.0, logical_width(),
current_estimates,
confidence );
}
}
uint64_t BinnedLiveGraph::advance( void )
{
unique_lock<mutex> ul { mutex_ };
const uint64_t now = timestamp();
const uint64_t now_bin = now / bin_width_ms_;
while ( current_bin_ < now_bin ) {
for ( unsigned int i = 0; i < value_this_bin_.size(); i++ ) {
double value = value_this_bin_[ i ] * multiplier_;
if ( rate_quantity_ ) {
value /= (bin_width_ms_ / 1000.0);
}
graph_.add_data_point( i,
(current_bin_ + 1) * bin_width_ms_ / 1000.0,
value );
initialize_new_bin_( bin_width_ms_, value_this_bin_[ i ] );
}
current_bin_++;
}
return now;
}
void BinnedLiveGraph::add_value_now( const unsigned int num, const unsigned int amount )
{
advance();
unique_lock<mutex> ul { mutex_ };
if ( value_this_bin_.at( num ) < 0 ) {
throw runtime_error( "BinnedLiveGraph: attempt to add to a default value" );
}
value_this_bin_.at( num ) += amount;
}
void BinnedLiveGraph::set_max_value_now( const unsigned int num, const unsigned int amount )
{
advance();
unique_lock<mutex> ul { mutex_ };
if ( value_this_bin_.at( num ) < 0 ) {
value_this_bin_.at( num ) = amount;
} else {
value_this_bin_.at( num ) = max( unsigned( value_this_bin_.at( num ) ), amount );
}
}
BinnedLiveGraph::~BinnedLiveGraph()
{
halt_ = true;
animation_thread_.join();
if ( animation_thread_exception_ != exception_ptr() ) {
try {
rethrow_exception( animation_thread_exception_ );
} catch ( const exception & e ) {
cerr << "BinnedLiveGraph exited from exception: ";
print_exception( e );
}
}
}
src/graphing/binned_livegraph.hh
+53
View File
@@ -0,0 +1,53 @@
/* -*-mode:c++; tab-width: 4; indent-tabs-mode: nil; c-basic-offset: 4 -*- */
#ifndef BINNED_LIVEGRAPH_HH
#define BINNED_LIVEGRAPH_HH
#include <vector>
#include <string>
#include <atomic>
#include <thread>
#include <exception>
#include <mutex>
#include <functional>
#include "graph.hh"
class BinnedLiveGraph
{
private:
Graph graph_;
unsigned int bin_width_ms_;
std::vector<int> value_this_bin_;
uint64_t current_bin_;
double multiplier_;
bool rate_quantity_;
uint64_t advance( void );
double logical_width( void ) const;
void animation_loop( void );
std::mutex mutex_;
std::atomic<bool> halt_;
std::exception_ptr animation_thread_exception_;
std::thread animation_thread_;
std::function<void(int,int&)> initialize_new_bin_;
public:
BinnedLiveGraph( const std::string & name, const Graph::StylesType & styles,
const std::string & y_label,
const double multiplier, const bool rate_quantity,
const unsigned int bin_width_ms,
const std::function<void(int,int&)> initialize_new_bin );
~BinnedLiveGraph();
void add_value_now( const unsigned int num, const unsigned int amount );
void set_max_value_now( const unsigned int num, const unsigned int amount );
};
#endif /* BINNED_LIVEGRAPH_HH */
src/graphing/cairo_objects.cc
+182
View File
@@ -0,0 +1,182 @@
#define CAIRO_HAS_XCB_SHM_FUNCTIONS 1
#include <stdexcept>
#include <mutex>
#include <cairo-xcb.h>
#include "cairo_objects.hh"
#include "display.hh"
using namespace std;
void Cairo::Surface::Deleter::operator() ( cairo_surface_t * x ) const
{
cairo_surface_destroy( x );
}
void Cairo::Context::Deleter::operator() ( cairo_t * x ) const
{
cairo_destroy( x );
}
void Cairo::Pattern::Deleter::operator() ( cairo_pattern_t * x ) const
{
cairo_pattern_destroy( x );
}
void Pango::Font::Deleter::operator() ( PangoFontDescription * x ) const
{
pango_font_description_free( x );
}
void Pango::Text::Deleter::operator() ( cairo_path_t * x ) const
{
cairo_path_destroy( x );
}
Cairo::Cairo( XPixmap & pixmap )
: surface_( pixmap ),
context_( surface_ )
{
check_error();
}
const pair<unsigned int, unsigned int> & Cairo::size( void ) const
{
return surface_.size;
}
Cairo::Surface::Surface( XPixmap & pixmap )
: size( pixmap.size() ),
surface( cairo_xcb_surface_create( pixmap.xcb_connection(),
pixmap.xcb_pixmap(),
pixmap.xcb_visual(),
size.first,
size.second ) )
{
check_error();
}
Cairo::Context::Context( Surface & surface )
: context( cairo_create( surface.surface.get() ) )
{
check_error();
}
void Cairo::Surface::check_error( void )
{
const cairo_status_t surface_result = cairo_surface_status( surface.get() );
if ( surface_result ) {
throw runtime_error( string( "cairo surface error: " ) + cairo_status_to_string( surface_result ) );
}
}
void Cairo::Context::check_error( void )
{
const cairo_status_t context_result = cairo_status( context.get() );
if ( context_result ) {
throw runtime_error( string( "cairo context error: " ) + cairo_status_to_string( context_result ) );
}
}
void Cairo::check_error( void )
{
context_.check_error();
surface_.check_error();
}
Pango::Pango( Cairo & cairo )
: context_( pango_cairo_create_context( cairo ) ),
layout_( pango_layout_new( *this ) )
{}
Pango::Font::Font( const string & description )
: font( pango_font_description_from_string( description.c_str() ) )
{}
void Pango::set_font( const Pango::Font & font )
{
pango_layout_set_font_description( *this, font );
}
mutex & global_pango_mutex( void )
{
static mutex global_pango_mutex_;
return global_pango_mutex_;
}
Pango::Text::Text( Cairo & cairo, Pango & pango, const Font & font, const string & text )
: path_(),
extent_( { 0, 0, 0, 0 } )
{
unique_lock<mutex> ul { global_pango_mutex() };
cairo_identity_matrix( cairo );
cairo_new_path( cairo );
pango.set_font( font );
pango_layout_set_text( pango, text.data(), text.size() );
pango_cairo_layout_path( cairo, pango );
path_.reset( cairo_copy_path( cairo ) );
/* get logical extents */
PangoRectangle logical;
pango_layout_get_extents( pango, nullptr, &logical );
extent_ = { logical.x / double( PANGO_SCALE ),
logical.y / double( PANGO_SCALE ),
logical.width / double( PANGO_SCALE ),
logical.height / double( PANGO_SCALE ) };
}
void Pango::Text::draw_centered_at( Cairo & cairo, const double x, const double y, const double max_width ) const
{
cairo_identity_matrix( cairo );
cairo_new_path( cairo );
Cairo::Extent<true> my_extent = extent().to_device( cairo );
double center_x = x - my_extent.x - my_extent.width / 2;
double center_y = y - my_extent.y - my_extent.height / 2;
if ( my_extent.width > max_width ) {
const double scale_factor = max_width / my_extent.width;
cairo_scale( cairo, scale_factor, scale_factor );
center_x = x - my_extent.x - my_extent.width * scale_factor / 2;
center_y = y - my_extent.y - my_extent.height * scale_factor / 2;
}
cairo_device_to_user( cairo, &center_x, &center_y );
cairo_translate( cairo, center_x, center_y );
cairo_append_path( cairo, path_.get() );
}
void Pango::Text::draw_centered_rotated_at( Cairo & cairo, const double x, const double y ) const
{
cairo_identity_matrix( cairo );
cairo_new_path( cairo );
cairo_rotate( cairo, - 3.1415926 / 2.0 );
Cairo::Extent<true> my_extent = extent().to_device( cairo );
double center_x = x - my_extent.x - my_extent.width / 2;
double center_y = y - my_extent.y - my_extent.height / 2;
cairo_device_to_user( cairo, &center_x, &center_y );
cairo_translate( cairo, center_x, center_y );
cairo_append_path( cairo, path_.get() );
}
Cairo::Pattern::Pattern( cairo_pattern_t * pattern )
: pattern_( pattern )
{
const cairo_status_t pattern_result = cairo_pattern_status( pattern_.get() );
if ( pattern_result ) {
throw runtime_error( string( "cairo pattern error: " ) + cairo_status_to_string( pattern_result ) );
}
}
src/graphing/cairo_objects.hh
+158
View File
@@ -0,0 +1,158 @@
#ifndef CAIRO_OBJECTS_HH
#define CAIRO_OBJECTS_HH
#define CAIRO_HAS_XCB_SHM_FUNCTIONS 1
#include <cairo.h>
#include <pango/pangocairo.h>
#include <memory>
#include <limits>
class XPixmap;
class Cairo
{
struct Surface
{
struct Deleter
{
void operator() ( cairo_surface_t * x ) const;
};
std::pair<unsigned int, unsigned int> size;
std::unique_ptr<cairo_surface_t, Deleter> surface;
Surface( XPixmap & pixmap );
void check_error( void );
} surface_;
struct Context
{
struct Deleter
{
void operator() ( cairo_t * x ) const;
};
std::unique_ptr<cairo_t, Deleter> context;
Context( Surface & surface );
void check_error( void );
} context_;
void check_error( void );
public:
Cairo( XPixmap & pixmap );
const std::pair<unsigned int, unsigned int> & size( void ) const;
operator cairo_t * () { return context_.context.get(); }
template <bool device_coordinates>
struct Extent
{
double x, y, width, height;
Extent<false> to_user( Cairo & cairo ) const
{
static_assert( device_coordinates == true,
"Extent::to_user() called but coordinates already in user-space" );
double x1 = x, x2 = x + width, y1 = y, y2 = y + height;
cairo_device_to_user( cairo, &x1, &y1 );
cairo_device_to_user( cairo, &x2, &y2 );
return Extent<false>( { x1, y1, x2 - x1, y2 - y1 } );
}
Extent<true> to_device( Cairo & cairo ) const
{
static_assert( device_coordinates == false,
"Extent::to_device() called but coordinates already in device-space" );
double x1 = x, x2 = x + width, y1 = y, y2 = y + height;
cairo_user_to_device( cairo, &x1, &y1 );
cairo_user_to_device( cairo, &x2, &y2 );
return Extent<true>( { x1, y1, x2 - x1, y2 - y1 } );
}
};
class Pattern
{
struct Deleter
{
void operator() ( cairo_pattern_t * x ) const;
};
std::unique_ptr<cairo_pattern_t, Deleter> pattern_;
public:
Pattern( cairo_pattern_t * pattern );
operator cairo_pattern_t * () { return pattern_.get(); }
};
};
template <class T>
struct PangoDelete {
void operator() ( T * x ) const
{
g_object_unref( x );
}
};
class Pango
{
std::unique_ptr<PangoContext, PangoDelete<PangoContext>> context_;
std::unique_ptr<PangoLayout, PangoDelete<PangoLayout>> layout_;
public:
Pango( Cairo & cairo );
operator PangoContext * () { return context_.get(); }
operator PangoLayout * () { return layout_.get(); }
struct Font
{
struct Deleter
{
void operator() ( PangoFontDescription * x ) const;
};
std::unique_ptr<PangoFontDescription, Deleter> font;
Font( const std::string & description );
operator const PangoFontDescription * () const { return font.get(); }
};
class Text
{
struct Deleter
{
void operator() ( cairo_path_t * x ) const;
};
std::unique_ptr<cairo_path_t, Deleter> path_;
Cairo::Extent<false> extent_;
public:
Text( Cairo & cairo, Pango & pango, const Font & font, const std::string & text );
const Cairo::Extent<false> & extent( void ) const { return extent_; }
void draw_centered_at( Cairo & cairo, const double x, const double y, const double max_width = std::numeric_limits<double>::max() ) const;
void draw_centered_rotated_at( Cairo & cairo, const double x, const double y ) const;
operator const cairo_path_t * () const { return path_.get(); }
};
void set_font( const Font & font );
};
#endif /* CAIRO_OBJECTS_HH */
src/graphing/display.cc
+240
View File
@@ -0,0 +1,240 @@
#include <stdexcept>
#include <iostream>
#include <cstdlib>
#include <xcb/present.h>
#include "display.hh"
using namespace std;
template <typename T>
inline T * notnull( const string & context, T * const x )
{
return x ? x : throw runtime_error( context + ": returned null pointer" );
}
struct free_deleter
{
template <typename T>
void operator() ( T * x ) const { free( x ); }
};
void XCBObject::check_noreply( const string & context, const xcb_void_cookie_t & cookie )
{
unique_ptr<xcb_generic_error_t, free_deleter> error { xcb_request_check( connection_.get(), cookie ) };
if ( error ) {
throw runtime_error( context + ": returned error " + to_string( error->error_code ) );
}
}
const xcb_screen_t * XCBObject::default_screen( void ) const
{
return notnull( "xcb_setup_roots_iterator",
xcb_setup_roots_iterator( notnull( "xcb_get_setup",
xcb_get_setup( connection_.get() ) ) ).data );
}
XCBObject::XCBObject( void )
: connection_( notnull( "xcb_connect", xcb_connect( nullptr, nullptr ) ),
[] ( xcb_connection_t * connection ) { xcb_disconnect( connection ); } )
{}
XCBObject::XCBObject( XCBObject & original )
: connection_( original.connection() )
{}
XCBObject::XCBObject( XCBObject && original )
: connection_( move( original.connection_ ) )
{}
XWindow::XWindow( const unsigned int width, const unsigned int height )
{
const auto screen = default_screen();
check_noreply( "xcb_create_window_checked",
xcb_create_window_checked( connection().get(),
XCB_COPY_FROM_PARENT,
window_,
screen->root,
0, 0, /* top-left corner of the window */
width, height,
0, /* border size */
XCB_WINDOW_CLASS_INPUT_OUTPUT,
screen->root_visual,
0, nullptr ) ); /* value_mask, value_list */
check_noreply( "xcb_present_select_input_checked",
xcb_present_select_input_checked( connection().get(),
complete_event_,
window_,
XCB_PRESENT_EVENT_MASK_COMPLETE_NOTIFY ) );
check_noreply( "xcb_present_select_input_checked",
xcb_present_select_input_checked( connection().get(),
idle_event_,
window_,
XCB_PRESENT_EVENT_MASK_IDLE_NOTIFY ) );
}
void XWindow::map( void ) {
check_noreply( "xcb_map_window_checked",
xcb_map_window_checked( connection().get(), window_ ) );
}
void XWindow::flush( void ) {
if ( xcb_flush( connection().get() ) <= 0 ) {
throw runtime_error( "xcb_flush: failed" );
}
}
XWindow::~XWindow( void )
{
try {
check_noreply( "xcb_destroy_window_checked",
xcb_destroy_window_checked( connection().get(), window_ ) );
} catch ( const exception & e ) {
cerr << e.what() << endl;
}
}
void XWindow::set_name( const string & name )
{
check_noreply( "xcb_change_property_checked",
xcb_change_property_checked( connection().get(),
XCB_PROP_MODE_REPLACE,
window_,
XCB_ATOM_WM_NAME,
XCB_ATOM_STRING,
8, /* 8-bit string */
name.length(),
name.data() ) );
}
xcb_visualtype_t * XWindow::xcb_visual( void )
{
/* get the visual ID of the window */
xcb_get_window_attributes_cookie_t cookie = xcb_get_window_attributes( connection().get(), window_ );
unique_ptr<xcb_get_window_attributes_reply_t, free_deleter> attributes { notnull( "xcb_get_attributes_reply",
xcb_get_window_attributes_reply( connection().get(), cookie, nullptr ) ) };
/* now find the corresponding visual */
const auto screen = default_screen();
for ( xcb_depth_iterator_t depth_it = xcb_screen_allowed_depths_iterator( screen );
depth_it.rem;
xcb_depth_next( &depth_it ) ) {
for ( xcb_visualtype_iterator_t visualtype_it = xcb_depth_visuals_iterator( depth_it.data );
visualtype_it.rem;
xcb_visualtype_next( &visualtype_it ) ) {
if ( attributes->visual == visualtype_it.data->visual_id ) {
return visualtype_it.data;
}
}
}
throw runtime_error( "xcb_visual: no matching visualtype_t found" );
}
pair<unsigned int, unsigned int> XWindow::size( void ) const
{
xcb_get_geometry_cookie_t cookie = xcb_get_geometry( connection().get(), window_ );
unique_ptr<xcb_get_geometry_reply_t, free_deleter> geometry { notnull( "xcb_get_geometry_reply",
xcb_get_geometry_reply( connection().get(), cookie, nullptr ) ) };
return make_pair( geometry->width, geometry->height );
}
XPixmap::XPixmap( XWindow & window )
: XCBObject( window ),
pixmap_( xcb_generate_id( connection().get() ) ),
visual_( window.xcb_visual() ),
size_( window.size() )
{
check_noreply( "xcb_create_pixmap_checked",
xcb_create_pixmap_checked( connection().get(),
default_screen()->root_depth,
pixmap_,
window.xcb_window(),
size_.first, size_.second ) );
}
XPixmap::XPixmap( XPixmap && other )
: XCBObject( move( other ) ),
pixmap_( other.pixmap_ ),
visual_( other.visual_ ),
size_( other.size_ )
{}
XPixmap & XPixmap::operator=( XPixmap && other )
{
this->~XPixmap();
new (this) XPixmap( move( other ) );
return *this;
}
XPixmap::~XPixmap()
{
if ( not connection() ) {
/* moved away */
return;
}
try {
check_noreply( "xcb_free_pixmap_checked",
xcb_free_pixmap_checked( connection().get(), pixmap_ ) );
} catch ( const exception & e ) {
cerr << e.what() << endl;
}
}
void XWindow::present( const XPixmap & pixmap, const unsigned int divisor, const unsigned int remainder )
{
while ( not complete_ ) {
event_loop();
}
check_noreply( "xcb_present_pixmap_checked",
xcb_present_pixmap_checked( connection().get(),
window_,
pixmap.xcb_pixmap(),
0, /* serial */
0, /* valid */
0, /* update */
0, 0, /* offsets */
0, /* target_crtc */
0, /* wait_fence */
0, /* idle_fence */
0, /* options */
0, /* target_msc */
divisor, /* divisor */
remainder, /* remainder */
0, /* notifies_len */
nullptr /* notifies */ ) );
complete_ = false;
idle_ = false;
while ( not idle_ ) {
event_loop();
}
}
void XWindow::event_loop( void )
{
xcb_generic_event_t * event = notnull( "xcb_wait_for_event",
xcb_wait_for_event( connection().get() ) );
if ( event->response_type == XCB_GE_GENERIC ) {
const uint16_t event_type = reinterpret_cast<xcb_ge_generic_event_t *>( event )->event_type;
if ( event_type == (complete_event_ & 0xffff) ) {
complete_ = true;
} else if ( event_type == (idle_event_ & 0xffff) ) {
idle_ = true;
} else {
throw runtime_error( "unexpected present event" );
}
} else {
// throw runtime_error( "unexpected event" );
// This is happening (but very rarely). Let's ignore for now. (KJW 1/23/2015)
}
}
src/graphing/display.hh
+101
View File
@@ -0,0 +1,101 @@
#ifndef DISPLAY_HH
#define DISPLAY_HH
#include <xcb/xcb.h>
#include <string>
#include <memory>
class XCBObject
{
public:
typedef std::shared_ptr<xcb_connection_t> connection_type;
XCBObject();
XCBObject( XCBObject & original );
XCBObject( XCBObject && original );
virtual ~XCBObject() {}
xcb_connection_t * xcb_connection( void ) { return connection_.get(); }
private:
connection_type connection_;
protected:
void check_noreply( const std::string & context, const xcb_void_cookie_t & cookie );
const xcb_screen_t * default_screen( void ) const;
const connection_type & connection( void ) const { return connection_; }
};
class XPixmap;
class XWindow : public XCBObject
{
private:
xcb_window_t window_ = xcb_generate_id( connection().get() );
uint32_t complete_event_ = xcb_generate_id( connection().get() );
uint32_t idle_event_ = xcb_generate_id( connection().get() );
bool complete_ = true, idle_ = true;
void event_loop( void );
public:
XWindow( const unsigned int width, const unsigned int height );
XWindow( XCBObject & original, const unsigned int width, const unsigned int height );
~XWindow();
/* set the name of the window */
void set_name( const std::string & name );
/* map the window on the screen */
void map( void );
/* present a pixmap */
void present( const XPixmap & pixmap, const unsigned int divisor, const unsigned int remainder );
/* flush XCB */
void flush( void );
/* get the window's size */
std::pair<unsigned int, unsigned int> size( void ) const;
/* get the underlying window */
const xcb_window_t & xcb_window( void ) const { return window_; }
/* get the window's visual */
xcb_visualtype_t * xcb_visual( void );
/* prevent copying */
XWindow( const XWindow & other ) = delete;
XWindow & operator=( const XWindow & other ) = delete;
};
class XPixmap : public XCBObject
{
private:
xcb_pixmap_t pixmap_;
xcb_visualtype_t * visual_;
std::pair<unsigned int, unsigned int> size_;
public:
XPixmap( XWindow & window );
~XPixmap();
/* get the underlying pixmap */
const xcb_pixmap_t & xcb_pixmap( void ) const { return pixmap_; }
/* get the pixmap's visual */
xcb_visualtype_t * xcb_visual( void ) const { return visual_; }
/* get the pixmap's size */
std::pair<unsigned int, unsigned int> size( void ) const { return size_; }
/* prevent copying */
XPixmap( const XPixmap & other ) = delete;
XPixmap & operator=( const XPixmap & other ) = delete;
/* allow moving */
XPixmap( XPixmap && other );
XPixmap & operator=( XPixmap && other );
};
#endif
src/graphing/graph.cc
+375
View File
@@ -0,0 +1,375 @@
#include <cmath>
#include <sstream>
#include <locale>
#include <numeric>
#include <limits>
#include <algorithm>
#include <cassert>
#include <iostream>
#include "graph.hh"
using namespace std;
Graph::GraphicContext::GraphicContext( XWindow & window )
: pixmap( window ),
cairo( pixmap ),
pango( cairo )
{}
Graph::GraphicContext & Graph::current_gc( void )
{
return gcs_[ current_gc_ ];
}
Graph::Graph( const unsigned int initial_width, const unsigned int initial_height,
const string & title,
const float min_y, const float max_y,
const StylesType & styles,
const string & x_label,
const string & y_label )
: window_( initial_width, initial_height ),
gcs_( { window_, window_, window_ } ),
current_gc_( 0 ),
tick_font_( "Open Sans Condensed Bold 20" ),
label_font_( "Open Sans Condensed Bold 20" ),
x_tick_labels_(),
y_tick_labels_(),
styles_( styles ),
data_points_( styles_.size() ),
x_label_( current_gc().cairo, current_gc().pango, label_font_, x_label ),
y_label_( current_gc().cairo, current_gc().pango, label_font_, y_label ),
info_string_(),
info_( current_gc().cairo, current_gc().pango, label_font_, "" ),
target_min_y_( min_y ),
target_max_y_( max_y ),
bottom_( min_y ),
top_( max_y ),
horizontal_fadeout_( cairo_pattern_create_linear( 0, 0, 190, 0 ) ),
data_mutex_()
{
cairo_pattern_add_color_stop_rgba( horizontal_fadeout_, 0.0, 1, 1, 1, 1 );
cairo_pattern_add_color_stop_rgba( horizontal_fadeout_, 0.67, 1, 1, 1, 1 );
cairo_pattern_add_color_stop_rgba( horizontal_fadeout_, 1.0, 1, 1, 1, 0 );
window_.set_name( title );
window_.map();
window_.flush();
}
static int to_int( const float x )
{
return static_cast<int>( lrintf( x ) );
}
bool Graph::blocking_draw( const float t, const float logical_width,
const vector<float> & current_values, const double current_weight )
{
unique_lock<mutex> ul { data_mutex_ }; /* going to read and write data_points_ */
for ( auto & line : data_points_ ) {
while ( (line.size() >= 2) and (line.front().first < t - logical_width - 1)
and (line.at( 1 ).first < t - logical_width - 1) ) {
line.pop_front();
}
}
const vector<deque<pair<float, float>>> data_points_snapshot = data_points_;
ul.unlock();
assert( data_points_snapshot.size() == current_values.size() );
assert( current_weight >= 0 );
assert( current_weight <= 1 );
/* autoscale graph -- but only lines (not filled areas) */
float max_value = numeric_limits<float>::min();
/* look at historical data points */
for ( unsigned int i = 0; i < data_points_snapshot.size(); i++ ) {
if ( get<4>( styles_.at( i ) ) ) { /* skip filled areas */
continue;
}
for ( const auto & point : data_points_snapshot.at( i ) ) {
if ( point.second > max_value ) {
max_value = point.second;
}
}
/* look at current/provisional data points? */
if ( current_weight > 0.4 ) {
if ( current_values.at( i ) > max_value ) {
max_value = current_values.at( i );
}
}
}
/* expansion? */
if ( max_value * 1.2 > target_max_y_ ) {
target_max_y_ = max( max_value * 1.4f, target_min_y_ + 1 );
}
/* contraction */
if ( max_value * 1.8 < target_max_y_ ) {
target_max_y_ = max( max_value * 1.6f, target_min_y_ + 1 );
}
/* set scale for this frame (with smoothing) */
top_ = top_ * .95 + target_max_y_ * 0.05;
bottom_ = bottom_ * 0.95 + target_min_y_ * 0.05;
/* get the current window size */
const auto window_size = window_.size();
/* do we need to resize? */
if ( window_size != current_gc().cairo.size() ) {
current_gc() = GraphicContext( window_ );
}
Cairo & cairo_ = current_gc().cairo;
Pango & pango_ = current_gc().pango;
/* start a new image */
cairo_new_path( cairo_ );
cairo_identity_matrix( cairo_ );
cairo_rectangle( cairo_, 0, 0, window_size.first, window_size.second );
cairo_set_source_rgba( cairo_, 1, 1, 1, 1 );
cairo_fill( cairo_ );
/* do we need to delete a label? */
while ( (not x_tick_labels_.empty()) and (x_tick_labels_.front().first < t - logical_width - 1) ) {
x_tick_labels_.pop_front();
}
/* do we need to make a new label? */
while ( x_tick_labels_.empty() or (x_tick_labels_.back().first < t + 1) ) { /* start when offscreen */
const int next_label = x_tick_labels_.empty() ? to_int( t ) : x_tick_labels_.back().first + 1;
/* add commas as appropriate */
stringstream ss;
ss.imbue( locale( "" ) );
ss << fixed << next_label;
x_tick_labels_.emplace_back( next_label, Pango::Text( cairo_, pango_, tick_font_, ss.str() ) );
}
/* draw the labels and vertical grid */
for ( const auto & x : x_tick_labels_ ) {
/* position the text in the window */
const double x_position = window_size.first - (t - x.first) * window_size.first / logical_width;
x.second.draw_centered_at( cairo_,
x_position,
window_size.second * 9.0 / 10.0,
0.85 * window_size.first / logical_width );
cairo_set_source_rgba( cairo_, 0, 0, 0.4, 1 );
cairo_fill( cairo_ );
/* draw vertical grid line */
cairo_identity_matrix( cairo_ );
cairo_set_line_width( cairo_, 2 );
cairo_move_to( cairo_, x_position, chart_height( bottom_, window_size.second ) );
cairo_line_to( cairo_, x_position, chart_height( top_, window_size.second ) );
cairo_set_source_rgba( cairo_, 0, 0, 0.4, 0.25 );
cairo_stroke( cairo_ );
}
/* draw the x-axis label */
x_label_.draw_centered_at( cairo_, 35 + window_size.first / 2, window_size.second * 9.6 / 10.0 );
cairo_set_source_rgba( cairo_, 0, 0, 0.4, 1 );
cairo_fill( cairo_ );
/* draw the data */
for ( unsigned int line_no = 0; line_no < data_points_snapshot.size(); line_no++ ) {
const auto & line = data_points_snapshot.at( line_no );
if ( line.empty() ) {
continue;
}
cairo_set_source_rgba( cairo_,
get<0>( styles_.at( line_no ) ),
get<1>( styles_.at( line_no ) ),
get<2>( styles_.at( line_no ) ),
get<3>( styles_.at( line_no ) ) );
bool pen_down = false;
pair<float, float> last_point;
for ( unsigned int i = 0; i < line.size(); i++ ) {
if ( pen_down ) {
if ( line[ i ].second >= 0 ) {
add_segment( t, line[ i ].first, line[ i ].second, logical_width );
last_point = line[ i ];
} else {
end_line( t, last_point.first, logical_width, get<4>( styles_.at( line_no ) ) );
pen_down = false;
}
} else if ( line[ i ].second >= 0 ) {
begin_line( t, line[ i ].first, line[ i ].second, logical_width );
last_point = line[ i ];
pen_down = true;
}
}
if ( pen_down ) {
if ( (line.back().second >= 0) and (current_values.at( line_no ) >= 0) ) {
add_segment( t, t,
current_weight * current_values.at( line_no ) + (1 - current_weight) * line.back().second,
logical_width );
end_line( t, line.front().first,
logical_width, get<4>( styles_.at( line_no ) ) );
} else {
end_line( t, line.front().first, logical_width, get<4>( styles_.at( line_no ) ) );
}
}
}
/* draw the y-axis labels */
double label_bottom = to_int( floor( bottom_ ) );
double label_top = to_int( ceil( top_ ) );
double label_spacing = 1.0/4.0;
while ( label_spacing < (label_top - label_bottom) / 6 ) {
label_spacing *= 2;
}
label_bottom = (label_bottom / label_spacing) * label_spacing;
label_top = (label_top / label_spacing) * label_spacing;
/* cull old labels */
{
auto it = y_tick_labels_.begin();
while ( it < y_tick_labels_.end() ) {
if ( it->intensity < 0.01 ) {
/* delete it */
auto it_next = it + 1;
y_tick_labels_.erase( it );
it = it_next;
} else {
it++;
}
}
}
/* find the labels we actually want on this frame */
vector<pair<double, bool>> labels_that_belong;
for ( double val = label_bottom; val <= label_top; val += label_spacing ) {
if ( project_height( val ) < 0 or project_height( val ) > 1 ) {
continue;
}
labels_that_belong.emplace_back( val, false );
}
/* adjust current labels as necessary */
for ( auto it = y_tick_labels_.begin(); it != y_tick_labels_.end(); it++ ) {
bool belongs = false;
for ( auto & y : labels_that_belong ) {
if ( it->height == y.first ) {
assert( y.second == false ); /* don't want duplicates */
y.second = true;
belongs = true;
break;
}
}
if ( belongs ) {
it->intensity = 0.95 * it->intensity + 0.05;
} else {
it->intensity = 0.95 * it->intensity;
}
}
/* add new labels if necessary */
for ( const auto & x : labels_that_belong ) {
if ( x.second ) {
/* already found */
continue;
}
stringstream ss;
ss.imbue( locale( "" ) );
ss << dec << x.first;
y_tick_labels_.emplace_back( YLabel( { x.first, Pango::Text( cairo_, pango_, label_font_, ss.str() ), 0.05 } ) );
}
/* draw the horizontal grid lines */
for ( const auto & x : y_tick_labels_ ) {
cairo_identity_matrix( cairo_ );
cairo_set_line_width( cairo_, 1 );
cairo_move_to( cairo_, 0, chart_height( x.height, window_size.second ) );
cairo_line_to( cairo_, window_size.first, chart_height( x.height, window_size.second ) );
cairo_set_source_rgba( cairo_, 0, 0, 0.4, 0.25 * x.intensity );
cairo_stroke( cairo_ );
}
/* draw a box to protect the tick labels */
cairo_new_path( cairo_ );
cairo_identity_matrix( cairo_ );
cairo_rectangle( cairo_, 0, 0, 190, window_size.second );
cairo_set_source( cairo_, horizontal_fadeout_ );
cairo_fill( cairo_ );
/* draw the info */
info_.draw_centered_at( cairo_, window_size.first / 2, 20, window_size.first - 40 );
cairo_set_source_rgba( cairo_, 0.4, 0, 0, 1 );
cairo_fill( cairo_ );
/* draw the y-axis label */
y_label_.draw_centered_rotated_at( cairo_, 25, window_size.second * .4375 );
cairo_set_source_rgba( cairo_, 0, 0, 0.4, 1 );
cairo_fill( cairo_ );
/* go through and paint all the labels */
for ( const auto & x : y_tick_labels_ ) {
x.text.draw_centered_at( cairo_, 100, chart_height( x.height, window_size.second ) );
cairo_set_source_rgba( cairo_, 0, 0, 0.4, x.intensity );
cairo_fill( cairo_ );
}
window_.present( current_gc().pixmap, gcs_.size(), current_gc_ );
current_gc_ = (current_gc_ + 1) % gcs_.size();
return false;
}
void Graph::begin_line( const float t, const float x, const float y, const float logical_width )
{
Cairo & cairo_ = current_gc().cairo;
const auto & window_size = cairo_.size();
cairo_identity_matrix( cairo_ );
cairo_set_line_width( cairo_, 3 );
const double x_position = window_size.first - (t - x) * window_size.first / logical_width;
cairo_move_to( cairo_, x_position, chart_height( y, window_size.second ) );
}
void Graph::add_segment( const float t, const float x, const float y, const float logical_width )
{
Cairo & cairo_ = current_gc().cairo;
const auto & window_size = cairo_.size();
const double x_position = window_size.first - (t - x) * window_size.first / logical_width;
cairo_line_to( cairo_, x_position, chart_height( y, window_size.second ) );
}
void Graph::end_line( const float t, const float x, const float logical_width, const bool fill )
{
Cairo & cairo_ = current_gc().cairo;
const auto & window_size = cairo_.size();
if ( fill ) {
/* fill the curve */
cairo_line_to( cairo_, window_size.first, chart_height( 0, window_size.second ) );
cairo_line_to( cairo_, window_size.first - (t - x) * window_size.first / logical_width,
chart_height( 0, window_size.second ) );
cairo_fill( cairo_ );
} else {
cairo_stroke( cairo_ );
}
}
src/graphing/graph.hh
+88
View File
@@ -0,0 +1,88 @@
#ifndef GRAPH_HH
#define GRAPH_HH
#include <deque>
#include <vector>
#include <mutex>
#include "display.hh"
#include "cairo_objects.hh"
class Graph
{
struct GraphicContext
{
XPixmap pixmap;
Cairo cairo;
Pango pango;
GraphicContext( XWindow & window );
};
XWindow window_;
std::array<GraphicContext, 3> gcs_;
unsigned int current_gc_;
GraphicContext & current_gc( void );
Pango::Font tick_font_;
Pango::Font label_font_;
struct YLabel
{
double height;
Pango::Text text;
float intensity;
};
std::deque<std::pair<int, Pango::Text>> x_tick_labels_;
std::vector<YLabel> y_tick_labels_;
std::vector<std::tuple<float, float, float, float, bool>> styles_;
std::vector<std::deque<std::pair<float, float>>> data_points_;
Pango::Text x_label_;
Pango::Text y_label_;
std::string info_string_;
Pango::Text info_;
float target_min_y_, target_max_y_;
float bottom_, top_;
float project_height( const float x ) const { return ( x - bottom_ ) / ( top_ - bottom_ ); }
float chart_height( const float x, const unsigned int window_height ) const
{
return (window_height - 40) * (.825*(1-project_height( x ))+.025) + (.825 * 40);
}
Cairo::Pattern horizontal_fadeout_;
std::mutex data_mutex_;
void begin_line( const float t, const float x, const float y, const float logical_width );
void add_segment( const float t, const float x, const float y, const float logical_width );
void end_line( const float t, const float x, const float logical_width, const bool fill );
public:
typedef std::vector<std::tuple<float, float, float, float, bool>> StylesType;
Graph( const unsigned int initial_width, const unsigned int initial_height,
const std::string & title,
const float min_y, const float max_y,
const StylesType & styles,
const std::string & x_label,
const std::string & y_label );
void add_data_point( const unsigned int num, const float t, const float y ) {
std::unique_lock<std::mutex> ul { data_mutex_ };
data_points_.at( num ).emplace_back( t, y );
}
bool blocking_draw( const float t, const float logical_width,
const std::vector<float> & current_values, const double current_weight );
std::pair<unsigned int, unsigned int> size( void ) const { return window_.size(); }
};
#endif /* GRAPH_HH */
src/http/Makefile.am
+16
View File
@@ -0,0 +1,16 @@
AM_CPPFLAGS = -I$(srcdir)/../util -I../protobufs $(CXX11_FLAGS)
AM_CXXFLAGS = $(PICKY_CXXFLAGS)
noinst_LIBRARIES = libhttp.a
libhttp_a_SOURCES = http_request_parser.hh \
http_request.hh http_request.cc \
http_response.hh http_response.cc \
http_header.hh http_header.cc \
http_response_parser.hh http_response_parser.cc \
tokenize.hh mime_type.hh mime_type.cc \
body_parser.hh \
chunked_parser.hh chunked_parser.cc \
http_message.hh http_message.cc \
http_message_sequence.hh \
backing_store.hh backing_store.cc
src/http/backing_store.cc
+47
View File
@@ -0,0 +1,47 @@
/* -*-mode:c++; tab-width: 4; indent-tabs-mode: nil; c-basic-offset: 4 -*- */
#include <iostream>
#include <string>
#include "util.hh"
#include "backing_store.hh"
#include "http_record.pb.h"
#include "temp_file.hh"
using namespace std;
HTTPDiskStore::HTTPDiskStore( const string & record_folder )
: record_folder_( record_folder ),
mutex_()
{}
void HTTPDiskStore::save( const HTTPResponse & response, const Address & server_address )
{
unique_lock<mutex> ul( mutex_ );
/* output file to write current request/response pair protobuf (user has all permissions) */
//UniqueFile file( record_folder_ + "save" );
std::cout << "before stripped: " << response.request().first_line() << std::endl;
auto stripped = strip_query(response.request().first_line());
std::cout << "after stripped: " << stripped << std::endl;
string hash = to_string(hash32(reinterpret_cast<const uint8_t*>(stripped.c_str()),stripped.size()));
UniqueFile file( record_folder_ + hash );
std::cout << "hash: " << hash << std::endl;
/* construct protocol buffer */
MahimahiProtobufs::RequestResponse output;
output.set_ip( server_address.ip() );
output.set_port( server_address.port() );
//output.set_scheme( (server_address.port() == 443)
output.set_scheme( (server_address.port() != 80)
? MahimahiProtobufs::RequestResponse_Scheme_HTTPS
: MahimahiProtobufs::RequestResponse_Scheme_HTTP );
std::cout << "save ip to disk: " << server_address.str() << std::endl;
output.mutable_request()->CopyFrom( response.request().toprotobuf() );
output.mutable_response()->CopyFrom( response.toprotobuf() );
if ( not output.SerializeToFileDescriptor( file.fd().fd_num() ) ) {
throw runtime_error( "save_to_disk: failure to serialize HTTP request/response pair" );
}
}
src/http/backing_store.hh
+32
View File
@@ -0,0 +1,32 @@
/* -*-mode:c++; tab-width: 4; indent-tabs-mode: nil; c-basic-offset: 4 -*- */
#ifndef BACKING_STORE_HH
#define BACKING_STORE_HH
#include <string>
#include <mutex>
#include "http_request.hh"
#include "http_response.hh"
#include "address.hh"
/* abstract base class to store an HTTP request/response from a particular server address */
class HTTPBackingStore
{
public:
virtual void save( const HTTPResponse & response, const Address & server_address ) = 0;
virtual ~HTTPBackingStore() {}
};
class HTTPDiskStore : public HTTPBackingStore
{
private:
std::string record_folder_;
std::mutex mutex_;
public:
HTTPDiskStore( const std::string & record_folder );
void save( const HTTPResponse & response, const Address & server_address ) override;
};
#endif /* BACKING_STORE_HH */
src/http/body_parser.hh
+39
View File
@@ -0,0 +1,39 @@
/* -*-mode:c++; tab-width: 4; indent-tabs-mode: nil; c-basic-offset: 4 -*- */
#ifndef BODY_PARSER_HH
#define BODY_PARSER_HH
class BodyParser
{
public:
/* possible return values from body parser:
- entire string belongs to body
- only some of string (0 bytes to n bytes) belongs to body */
virtual std::string::size_type read( const std::string & str ) = 0;
/* does message become complete upon EOF in body? */
virtual bool eof( void ) const = 0;
virtual ~BodyParser() {}
};
/* used for RFC 2616 4.4 "rule 5" responses -- terminated only by EOF */
class Rule5BodyParser : public BodyParser
{
public:
/* all of buffer always belongs to body */
std::string::size_type read( const std::string & ) override
{
return std::string::npos;
}
/* does message become complete upon EOF in body? */
/* when there was no content-length header on a response, answer is yes */
bool eof( void ) const override
{
return true;
}
};
#endif /* BODY_PARSER_HH */
src/http/chunked_parser.cc
+123
View File
@@ -0,0 +1,123 @@
/* -*-mode:c++; tab-width: 4; indent-tabs-mode: nil; c-basic-offset: 4 -*- */
#include <cassert>
#include "ezio.hh"
#include "chunked_parser.hh"
using namespace std;
/* Take a chunk header and parse it assuming no folding */
uint32_t ChunkedBodyParser::get_chunk_size( const string & chunk_hdr ) const
{
/* Check that the chunk header ends with a CRLF */
assert(chunk_hdr.substr( chunk_hdr.length() - 2, 2 ) == "\r\n" );
/* If there are chunk extensions, ';' terminates chunk size */
auto pos = chunk_hdr.find( ";" );
/* There are no ';'s, and hence no chunk externsions, CRLF terminates chunk size */
if ( pos == string::npos ) {
pos = chunk_hdr.find( "\r\n" );
}
/* Can't be npos even now */
assert( pos != string::npos );
/* Parse hex string, after removing trailing spaces (RFC 2616 Section 2.1) */
auto hex_string = chunk_hdr.substr( 0, pos );
auto space_loc = hex_string.find( " " );
if ( space_loc != string::npos ) hex_string.erase( space_loc, hex_string.length() - space_loc );
return myatoi( hex_string, 16 );
}
string::size_type ChunkedBodyParser::read( const std::string & input_buffer )
{
parser_buffer_ += input_buffer;
while ( !parser_buffer_.empty() ) {
switch (state_) {
case CHUNK_HDR: {
auto it = parser_buffer_.find( "\r\n" );
if (it != string::npos) {
/* if you have CRLF, get chunk size & transition to CHUNK/TRAILER */
current_chunk_size_ = get_chunk_size( parser_buffer_.substr( 0, it + 2 ) );
/* Transition appropriately */
state_ = ( current_chunk_size_ == 0 ) ? TRAILER : CHUNK;
/* shrink parser_buffer_ */
parsed_so_far_ += ( it + 2 );
parser_buffer_ = parser_buffer_.substr( it + 2 );
break;
} else {
/* if you haven't seen a CRLF so far, do nothing */
acked_so_far_ += input_buffer.length();
return string::npos;
}
}
case CHUNK: {
if ( parser_buffer_.length() >= current_chunk_size_ + 2 ) {
/* accumulated enough bytes, check CRLF at the end of the chunk */
assert( parser_buffer_.substr( current_chunk_size_, 2 ) == "\r\n" );
/* Transition to next state */
state_ = CHUNK_HDR;
/* shrink parser_buffer_ */
parsed_so_far_ += current_chunk_size_ + 2;
parser_buffer_ = parser_buffer_.substr( current_chunk_size_ + 2 );
break;
} else {
/* Haven't seen enough bytes so far, do nothing */
acked_so_far_ += input_buffer.length();
return string::npos;
}
}
case TRAILER: {
if ( trailers_enabled_) {
/* We need two consecutive CRLFs */
return compute_ack_size( parser_buffer_,
"\r\n\r\n",
input_buffer.length() );
} else {
/* We need only one CRLF now */
return compute_ack_size( parser_buffer_,
"\r\n",
input_buffer.length() );
}
}
default: {
assert( false );
return false;
}
}
}
acked_so_far_ += input_buffer.length();
return string::npos;
}
/*
Computes the acknowledgement from the BodyParser to its caller,
telling it how much of the current input_buffer has been
successfully parsed.
*/
string::size_type ChunkedBodyParser::compute_ack_size( const string & haystack,
const string & needle,
const string::size_type input_size )
{
auto loc = haystack.find( needle );
if ( loc != string::npos ) {
/* Found it, eat up the whole buffer */
parsed_so_far_ += loc + needle.length();
assert( parsed_so_far_ > acked_so_far_ );
return ( parsed_so_far_ - acked_so_far_ );
} else {
/* Find unacknowledged buffer so far, ack it, and be done */
acked_so_far_ += input_size;
return loc;
}
}
src/http/chunked_parser.hh
+32
View File
@@ -0,0 +1,32 @@
/* -*-mode:c++; tab-width: 4; indent-tabs-mode: nil; c-basic-offset: 4 -*- */
#ifndef CHUNKED_BODY_PARSER_HH
#define CHUNKED_BODY_PARSER_HH
#include "body_parser.hh"
#include "exception.hh"
class ChunkedBodyParser : public BodyParser
{
private:
std::string::size_type compute_ack_size(const std::string & haystack,
const std::string & needle,
std::string::size_type input_size);
uint32_t get_chunk_size(const std::string & chunk_hdr) const;
std::string parser_buffer_ {""};
uint32_t current_chunk_size_ {0};
std::string::size_type acked_so_far_ {0};
std::string::size_type parsed_so_far_ {0};
enum {CHUNK_HDR, CHUNK, TRAILER} state_ {CHUNK_HDR};
const bool trailers_enabled_ {false};
public:
std::string::size_type read( const std::string & ) override;
/* Follow item 2, Section 4.4 of RFC 2616 */
bool eof( void ) const override { return true; }
ChunkedBodyParser(bool t_trailers_enabled) : trailers_enabled_( t_trailers_enabled ) {}
};
#endif /* CHUNKED_BODY_PARSER_HH */
src/http/http_header.cc
+56
View File
@@ -0,0 +1,56 @@
/* -*-mode:c++; tab-width: 4; indent-tabs-mode: nil; c-basic-offset: 4 -*- */
#include <string>
#include <assert.h>
#include "http_header.hh"
#include "exception.hh"
#include "ezio.hh"
using namespace std;
/* parse a header line into a key and a value */
HTTPHeader::HTTPHeader( const string & buf )
: key_(), value_()
{
const string separator = ":";
/* step 1: does buffer contain colon? */
size_t colon_location = buf.find( separator );
if ( colon_location == std::string::npos ) {
fprintf( stderr, "Buffer: %s\n", buf.c_str() );
throw runtime_error( "HTTPHeader: buffer does not contain colon" );
}
/* step 2: split buffer */
key_ = buf.substr( 0, colon_location );
string value_temp = buf.substr( colon_location + separator.size() );
/* strip whitespace */
size_t first_nonspace = value_temp.find_first_not_of( " " );
if ( first_nonspace == std::string::npos ) { /* handle case where value is only space */
value_ = value_temp;
} else {
value_ = value_temp.substr( first_nonspace );
}
/*
fprintf( stderr, "Got header. key=[[%s]] value = [[%s]]\n",
key_.c_str(), value_.c_str() );
*/
}
HTTPHeader::HTTPHeader( const MahimahiProtobufs::HTTPHeader & proto )
: key_( proto.key() ), value_( proto.value() )
{
}
MahimahiProtobufs::HTTPHeader HTTPHeader::toprotobuf( void ) const
{
MahimahiProtobufs::HTTPHeader ret;
ret.set_key( key_ );
ret.set_value( value_ );
return ret;
}
src/http/http_header.hh
+27
View File
@@ -0,0 +1,27 @@
/* -*-mode:c++; tab-width: 4; indent-tabs-mode: nil; c-basic-offset: 4 -*- */
#ifndef HTTP_HEADER_HH
#define HTTP_HEADER_HH
#include <string>
#include "http_record.pb.h"
class HTTPHeader
{
private:
std::string key_, value_;
public:
HTTPHeader( const std::string & buf );
const std::string & key( void ) const { return key_; }
const std::string & value( void ) const { return value_; }
std::string str( void ) const { return key_ + ": " + value_; }
HTTPHeader( const MahimahiProtobufs::HTTPHeader & proto );
MahimahiProtobufs::HTTPHeader toprotobuf( void ) const;
};
#endif /* HTTP_HEADER_HH */
src/http/http_message.cc
+201
View File
@@ -0,0 +1,201 @@
/* -*-mode:c++; tab-width: 4; indent-tabs-mode: nil; c-basic-offset: 4 -*- */
#include "http_message.hh"
#include "exception.hh"
#include "http_record.pb.h"
using namespace std;
/* methods called by an external parser */
void HTTPMessage::set_first_line( const string & str )
{
assert( state_ == FIRST_LINE_PENDING );
first_line_ = str;
state_ = HEADERS_PENDING;
}
void HTTPMessage::add_header( const std::string & str )
{
assert( state_ == HEADERS_PENDING );
headers_.emplace_back( str );
}
void HTTPMessage::done_with_headers( void )
{
assert( state_ == HEADERS_PENDING );
state_ = BODY_PENDING;
calculate_expected_body_size();
}
void HTTPMessage::set_expected_body_size( const bool is_known, const size_t value )
{
assert( state_ == BODY_PENDING );
expected_body_size_ = make_pair( is_known, value );
}
size_t HTTPMessage::read_in_body( const std::string & str )
{
assert( state_ == BODY_PENDING );
if ( body_size_is_known() ) {
/* body size known in advance */
assert( body_.size() <= expected_body_size() );
const size_t amount_to_append = min( expected_body_size() - body_.size(),
str.size() );
body_.append( str.substr( 0, amount_to_append ) );
if ( body_.size() == expected_body_size() ) {
state_ = COMPLETE;
}
return amount_to_append;
} else {
/* body size not known in advance */
return read_in_complex_body( str );
}
}
void HTTPMessage::eof( void )
{
switch ( state() ) {
case FIRST_LINE_PENDING:
return;
case HEADERS_PENDING:
throw runtime_error( "HTTPMessage: EOF received in middle of headers" );
case BODY_PENDING:
if ( eof_in_body() ) {
state_ = COMPLETE;
}
break;
case COMPLETE:
assert( false ); /* nobody should be calling methods on a complete message */
return;
}
}
bool HTTPMessage::body_size_is_known( void ) const
{
assert( state_ > HEADERS_PENDING );
return expected_body_size_.first;
}
size_t HTTPMessage::expected_body_size( void ) const
{
assert( body_size_is_known() );
return expected_body_size_.second;
}
/* locale-insensitive ASCII conversion */
static char http_to_lower( char c )
{
const char diff = 'A' - 'a';
if ( c >= 'A' and c <= 'Z' ) {
c -= diff;
}
return c;
}
static string strip_initial_whitespace( const string & str )
{
size_t first_nonspace = str.find_first_not_of( " " );
if ( first_nonspace == std::string::npos ) {
return "";
} else {
return str.substr( first_nonspace );
}
}
/* check if two strings are equivalent per HTTP 1.1 comparison (case-insensitive) */
bool HTTPMessage::equivalent_strings( const string & a, const string & b )
{
const string new_a = strip_initial_whitespace( a ),
new_b = strip_initial_whitespace( b );
if ( new_a.size() != new_b.size() ) {
return false;
}
for ( auto it_a = new_a.begin(), it_b = new_b.begin(); it_a < new_a.end(); it_a++, it_b++ ) {
if ( http_to_lower( *it_a ) != http_to_lower( *it_b ) ) {
return false;
}
}
return true;
}
bool HTTPMessage::has_header( const string & header_name ) const
{
for ( const auto & header : headers_ ) {
/* canonicalize header name per RFC 2616 section 2.1 */
if ( equivalent_strings( header.key(), header_name ) ) {
return true;
}
}
return false;
}
const string & HTTPMessage::get_header_value( const std::string & header_name ) const
{
for ( const auto & header : headers_ ) {
/* canonicalize header name per RFC 2616 section 2.1 */
if ( equivalent_strings( header.key(), header_name ) ) {
return header.value();
}
}
throw runtime_error( "HTTPMessage header not found: " + header_name );
}
/* serialize the request or response as one string */
std::string HTTPMessage::str( void ) const
{
assert( state_ == COMPLETE );
/* start with first line */
string ret( first_line_ + CRLF );
/* iterate through headers and add "key: value\r\n" to request */
for ( const auto & header : headers_ ) {
ret.append( header.str() + CRLF );
}
/* blank line between headers and body */
ret.append( CRLF );
/* add body to request */
ret.append( body_ );
return ret;
}
MahimahiProtobufs::HTTPMessage HTTPMessage::toprotobuf( void ) const
{
assert( state_ == COMPLETE );
MahimahiProtobufs::HTTPMessage ret;
ret.set_first_line( first_line_ );
for ( const auto & header : headers_ ) {
ret.add_header()->CopyFrom( header.toprotobuf() );
}
ret.set_body( body_ );
return ret;
}
HTTPMessage::HTTPMessage( const MahimahiProtobufs::HTTPMessage & proto )
: first_line_( proto.first_line() ),
body_( proto.body() ),
state_( COMPLETE )
{
for ( const auto header : proto.header() ) {
headers_.emplace_back( header );
}
}
src/http/http_message.hh
+85
View File
@@ -0,0 +1,85 @@
/* -*-mode:c++; tab-width: 4; indent-tabs-mode: nil; c-basic-offset: 4 -*- */
#ifndef HTTP_MESSAGE_HH
#define HTTP_MESSAGE_HH
#include <string>
#include <vector>
#include "http_header.hh"
#include "http_record.pb.h"
enum HTTPMessageState { FIRST_LINE_PENDING, HEADERS_PENDING, BODY_PENDING, COMPLETE };
/* helper for parsers */
static const std::string CRLF = "\r\n";
class HTTPMessage
{
private:
/* first member of pair specifies whether body size is known in advance,
and second member is size (if known in advance) */
/* this is calculated by calculate_expected_body_size() */
std::pair< bool, size_t > expected_body_size_ { false, -1 };
/* calculating the body size must be implemented by request or response */
virtual void calculate_expected_body_size( void ) = 0;
/* bodies with size not known in advance must be handled by subclass */
virtual size_t read_in_complex_body( const std::string & str ) = 0;
/* does message become complete upon EOF in body? */
virtual bool eof_in_body( void ) const = 0;
protected:
/* request line or status line */
std::string first_line_ {};
/* request/response headers */
std::vector< HTTPHeader > headers_ {};
/* body may be empty */
std::string body_ {};
/* state of an in-progress request or response */
HTTPMessageState state_ { FIRST_LINE_PENDING };
/* used by subclasses to set the expected body size */
void set_expected_body_size( const bool is_known, const size_t value = -1 );
public:
HTTPMessage() {}
virtual ~HTTPMessage() {}
/* methods called by an external parser */
void set_first_line( const std::string & str );
void add_header( const std::string & str );
void done_with_headers( void );
size_t read_in_body( const std::string & str );
void eof( void );
/* getters */
bool body_size_is_known( void ) const;
size_t expected_body_size( void ) const;
const HTTPMessageState & state( void ) const { return state_; }
const std::string & first_line( void ) const { return first_line_; }
/* troll through the headers */
bool has_header( const std::string & header_name ) const;
const std::string & get_header_value( const std::string & header_name ) const;
/* serialize the request or response as one string */
std::string str( void ) const;
/* return complete request or response as http_message protobuf */
MahimahiProtobufs::HTTPMessage toprotobuf( void ) const;
/* compare two strings for (case-insensitive) equality,
in ASCII without sensitivity to locale */
static bool equivalent_strings( const std::string & a, const std::string & b );
/* construct from protobuf */
HTTPMessage( const MahimahiProtobufs::HTTPMessage & proto );
};
#endif /* HTTP_MESSAGE_HH */
src/http/http_message_sequence.hh
+154
View File
@@ -0,0 +1,154 @@
/* -*-mode:c++; tab-width: 4; indent-tabs-mode: nil; c-basic-offset: 4 -*- */
#ifndef HTTP_MESSAGE_SEQUENCE
#define HTTP_MESSAGE_SEQUENCE
#include <string>
#include <queue>
#include "http_message.hh"
template <class MessageType>
class HTTPMessageSequence
{
private:
class InternalBuffer
{
private:
std::string buffer_ {};
public:
bool have_complete_line( void ) const;
std::string get_and_pop_line( void );
void pop_bytes( const size_t n );
bool empty( void ) const { return buffer_.empty(); }
void append( const std::string & str ) { buffer_.append( str ); }
const std::string & str( void ) const { return buffer_; }
};
/* bytes that haven't been parsed yet */
InternalBuffer buffer_ {};
/* complete messages ready to go */
std::queue< MessageType > complete_messages_ {};
/* one loop through the parser */
/* returns whether to continue */
bool parsing_step( void );
/* what to do to create a new message.
must be implemented by subclass */
virtual void initialize_new_message( void ) = 0;
protected:
/* the current message we're working on */
MessageType message_in_progress_ {};
public:
HTTPMessageSequence() {}
virtual ~HTTPMessageSequence() {}
/* must accept all of buf */
void parse( const std::string & buf );
/* getters */
bool empty( void ) const { return complete_messages_.empty(); }
const MessageType & front( void ) const { return complete_messages_.front(); }
/* pop one request */
void pop( void ) { complete_messages_.pop(); }
};
template <class MessageType>
bool HTTPMessageSequence<MessageType>::InternalBuffer::have_complete_line( void ) const
{
size_t first_line_ending = buffer_.find( CRLF );
return first_line_ending != std::string::npos;
}
template <class MessageType>
std::string HTTPMessageSequence<MessageType>::InternalBuffer::get_and_pop_line( void )
{
size_t first_line_ending = buffer_.find( CRLF );
assert( first_line_ending != std::string::npos );
std::string first_line( buffer_.substr( 0, first_line_ending ) );
pop_bytes( first_line_ending + CRLF.size() );
return first_line;
}
template <class MessageType>
void HTTPMessageSequence<MessageType>::InternalBuffer::pop_bytes( const size_t num )
{
assert( buffer_.size() >= num );
buffer_.replace( 0, num, std::string() );
}
template <class MessageType>
bool HTTPMessageSequence<MessageType>::parsing_step( void )
{
switch ( message_in_progress_.state() ) {
case FIRST_LINE_PENDING:
/* do we have a complete line? */
if ( not buffer_.have_complete_line() ) { return false; }
/* supply status line to request/response initialization routine */
initialize_new_message();
message_in_progress_.set_first_line( buffer_.get_and_pop_line() );
return true;
case HEADERS_PENDING:
/* do we have a complete line? */
if ( not buffer_.have_complete_line() ) { return false; }
/* is line blank? */
{
std::string line( buffer_.get_and_pop_line() );
if ( line.empty() ) {
message_in_progress_.done_with_headers();
} else {
message_in_progress_.add_header( line );
}
}
return true;
case BODY_PENDING:
{
size_t bytes_read = message_in_progress_.read_in_body( buffer_.str() );
assert( bytes_read == buffer_.str().size() or message_in_progress_.state() == COMPLETE );
buffer_.pop_bytes( bytes_read );
}
return message_in_progress_.state() == COMPLETE;
case COMPLETE:
complete_messages_.emplace( std::move( message_in_progress_ ) );
message_in_progress_ = MessageType();
return true;
}
assert( false );
return false;
}
template <class MessageType>
void HTTPMessageSequence<MessageType>::parse( const std::string & buf )
{
if ( buf.empty() ) { /* EOF */
message_in_progress_.eof();
}
/* append buf to internal buffer */
buffer_.append( buf );
/* parse as much as we can */
while ( parsing_step() ) {}
}
#endif /* HTTP_MESSAGE_SEQUENCE */
src/http/http_request.cc
+55
View File
@@ -0,0 +1,55 @@
/* -*-mode:c++; tab-width: 4; indent-tabs-mode: nil; c-basic-offset: 4 -*- */
#include "http_request.hh"
#include "exception.hh"
#include "ezio.hh"
using namespace std;
void HTTPRequest::calculate_expected_body_size( void )
{
assert( state_ == BODY_PENDING );
if ( first_line_.substr( 0, 4 ) == "GET "
or first_line_.substr( 0, 5 ) == "HEAD " ) {
set_expected_body_size( true, 0 );
} else if ( first_line_.substr( 0, 5 ) == "POST " ) {
if ( !has_header( "Content-Length" ) ) {
throw runtime_error( "HTTPRequest: does not support chunked requests" );
}
set_expected_body_size( true, myatoi( get_header_value( "Content-Length" ) ) );
} else if ( first_line_.substr( 0, 8 ) == "OPTIONS " ) {
if ( has_header( "Transfer-Encoding" ) && get_header_value( "Transfer-Encoding" ) == "chunked" ) {
throw runtime_error( "HTTPRequest: does not support chunked requests" );
}
size_t content_length = 0;
if ( has_header( "Content-Length" ) ) {
content_length = myatoi( get_header_value( "Content-Length" ) );
}
set_expected_body_size( true, content_length );
} else {
throw runtime_error( "Cannot handle HTTP method: " + first_line_ );
}
}
size_t HTTPRequest::read_in_complex_body( const std::string & )
{
/* we don't support complex bodies */
throw runtime_error( "HTTPRequest: does not support chunked requests" );
}
bool HTTPRequest::eof_in_body( void ) const
{
throw runtime_error( "HTTPRequest: got EOF in middle of body" );
}
bool HTTPRequest::is_head( void ) const
{
assert( state_ > FIRST_LINE_PENDING );
/* RFC 2616 5.1.1 says "The method is case-sensitive." */
return first_line_.substr( 0, 5 ) == "HEAD ";
}
src/http/http_request.hh
+26
View File
@@ -0,0 +1,26 @@
/* -*-mode:c++; tab-width: 4; indent-tabs-mode: nil; c-basic-offset: 4 -*- */
#ifndef HTTP_REQUEST_HH
#define HTTP_REQUEST_HH
#include "http_message.hh"
class HTTPRequest : public HTTPMessage
{
private:
/* for a request, will always be known */
void calculate_expected_body_size( void ) override;
/* we have no complex bodies */
size_t read_in_complex_body( const std::string & str ) override;
/* connection closed while body was pending */
bool eof_in_body( void ) const override;
public:
bool is_head( void ) const;
using HTTPMessage::HTTPMessage;
};
#endif /* HTTP_REQUEST_HH */
src/http/http_request_parser.hh
+15
View File
@@ -0,0 +1,15 @@
/* -*-mode:c++; tab-width: 4; indent-tabs-mode: nil; c-basic-offset: 4 -*- */
#ifndef HTTP_REQUEST_PARSER_HH
#define HTTP_REQUEST_PARSER_HH
#include "http_message_sequence.hh"
#include "http_request.hh"
class HTTPRequestParser : public HTTPMessageSequence<HTTPRequest>
{
private:
void initialize_new_message( void ) override {}
};
#endif /* HTTP_REQUEST_PARSER_HH */
src/http/http_response.cc
+101
View File
@@ -0,0 +1,101 @@
/* -*-mode:c++; tab-width: 4; indent-tabs-mode: nil; c-basic-offset: 4 -*- */
#include "tokenize.hh"
#include "http_response.hh"
#include "exception.hh"
#include "ezio.hh"
#include "mime_type.hh"
#include "chunked_parser.hh"
using namespace std;
string HTTPResponse::status_code( void ) const
{
assert( state_ > FIRST_LINE_PENDING );
auto tokens = split( first_line_, " " );
if ( tokens.size() < 3 ) {
throw runtime_error( "HTTPResponse: Invalid status line: " + first_line_ );
}
return tokens.at( 1 );
}
void HTTPResponse::calculate_expected_body_size( void )
{
assert( state_ == BODY_PENDING );
/* implement rules of RFC 2616 section 4.4 ("Message Length") */
if ( status_code().at( 0 ) == '1'
or status_code() == "204"
or status_code() == "304"
or request_.is_head() ) {
/* Rule 1: size known to be zero */
set_expected_body_size( true, 0 );
} else if ( has_header( "Transfer-Encoding" )
and equivalent_strings( split( get_header_value( "Transfer-Encoding" ), "," ).back(),
"chunked" ) ) {
/* Rule 2: size dictated by chunked encoding */
/* Rule 2 is a bit ambiguous, but we think section 3.6 makes this acceptable */
set_expected_body_size( false );
/* Create body_parser_ with trailers_enabled if requied (RFC 2616 section 14.40) */
body_parser_ = unique_ptr< BodyParser >( new ChunkedBodyParser( has_header( "Trailer" ) ) );
} else if ( (not has_header( "Transfer-Encoding" ) )
and has_header( "Content-Length" ) ) {
/* Rule 3: content-length header present to specify size */
set_expected_body_size( true, myatoi( get_header_value( "Content-Length" ) ) );
} else if ( has_header( "Content-Type" )
and equivalent_strings( MIMEType( get_header_value( "Content-Type" ) ).type(),
"multipart/byteranges" ) ) {
/* Rule 4 */
set_expected_body_size( false );
throw runtime_error( "HTTPResponse: unsupported multipart/byteranges without Content-Length" );
} else {
/* Rule 5 */
set_expected_body_size( false );
body_parser_ = unique_ptr< BodyParser >( new Rule5BodyParser() );
}
}
size_t HTTPResponse::read_in_complex_body( const std::string & str )
{
assert( state_ == BODY_PENDING );
assert( body_parser_ );
auto amount_parsed = body_parser_->read( str );
if ( amount_parsed == std::string::npos ) {
/* all of it belongs to the body */
body_.append( str );
return str.size();
} else {
/* body is now complete */
body_.append( str.substr( 0, amount_parsed ) );
state_ = COMPLETE;
return amount_parsed;
}
}
bool HTTPResponse::eof_in_body( void ) const
{
/* complex bodies sometimes allow an EOF to terminate the body */
if ( body_parser_ ) {
return body_parser_->eof();
} else {
throw runtime_error( "HTTPResponse: got EOF in middle of body" );
}
}
void HTTPResponse::set_request( const HTTPRequest & request )
{
assert( state_ == FIRST_LINE_PENDING );
request_ = request;
}
src/http/http_response.hh
+33
View File
@@ -0,0 +1,33 @@
/* -*-mode:c++; tab-width: 4; indent-tabs-mode: nil; c-basic-offset: 4 -*- */
#ifndef HTTP_RESPONSE_HH
#define HTTP_RESPONSE_HH
#include <memory>
#include "http_message.hh"
#include "body_parser.hh"
#include "http_request.hh"
class HTTPResponse : public HTTPMessage
{
private:
std::string status_code( void ) const;
HTTPRequest request_ {};
/* required methods */
void calculate_expected_body_size( void ) override;
size_t read_in_complex_body( const std::string & str ) override;
bool eof_in_body( void ) const override;
std::unique_ptr< BodyParser > body_parser_ { nullptr };
public:
void set_request( const HTTPRequest & request );
const HTTPRequest & request( void ) const { return request_; }
using HTTPMessage::HTTPMessage;
};
#endif /* HTTP_RESPONSE_HH */
src/http/http_response_parser.cc
+27
View File
@@ -0,0 +1,27 @@
/* -*-mode:c++; tab-width: 4; indent-tabs-mode: nil; c-basic-offset: 4 -*- */
#include <string>
#include <cassert>
#include "http_response_parser.hh"
#include "exception.hh"
#include "http_response.hh"
using namespace std;
void HTTPResponseParser::initialize_new_message( void )
{
/* do we have a request that we can match this response up with? */
if ( requests_.empty() ) {
throw runtime_error( "HTTPResponseParser: response without matching request" );
}
message_in_progress_.set_request( requests_.front() );
requests_.pop();
}
void HTTPResponseParser::new_request_arrived( const HTTPRequest & request )
{
requests_.push( request );
}
src/http/http_response_parser.hh
+22
View File
@@ -0,0 +1,22 @@
/* -*-mode:c++; tab-width: 4; indent-tabs-mode: nil; c-basic-offset: 4 -*- */
#ifndef HTTP_RESPONSE_PARSER_HH
#define HTTP_RESPONSE_PARSER_HH
#include "http_message_sequence.hh"
#include "http_response.hh"
#include "http_request.hh"
class HTTPResponseParser : public HTTPMessageSequence<HTTPResponse>
{
private:
/* Need this to handle RFC 2616 section 4.4 rule 1 */
std::queue< HTTPRequest > requests_ {};
void initialize_new_message( void ) override;
public:
void new_request_arrived( const HTTPRequest & request );
};
#endif /* HTTP_RESPONSE_PARSER_HH */
src/http/mime_type.cc
+18
View File
@@ -0,0 +1,18 @@
#include "mime_type.hh"
#include "tokenize.hh"
#include "exception.hh"
using namespace std;
MIMEType::MIMEType( const string & content_type )
: type_(), parameters_()
{
auto type_and_parameters = split( content_type, ";" );
if ( type_and_parameters.size() == 0
or type_and_parameters.at( 0 ).empty() ) {
throw runtime_error( "MIMEType: invalid MIME media-type string" );
}
type_ = type_and_parameters.at( 0 );
/* XXX don't parse the parameters for now */
}
src/http/mime_type.hh
+19
View File
@@ -0,0 +1,19 @@
#ifndef MIME_TYPE_HH
#define MIME_TYPE_HH
#include <vector>
#include <string>
class MIMEType
{
private:
std::string type_;
std::vector< std::pair< std::string, std::string > > parameters_;
public:
MIMEType( const std::string & content_type );
const std::string & type( void ) const { return type_; }
};
#endif /* MIME_TYPE_HH */
src/http/tokenize.hh
+41
View File
@@ -0,0 +1,41 @@
/* -*-mode:c++; tab-width: 4; indent-tabs-mode: nil; c-basic-offset: 4 -*- */
#ifndef TOKENIZE_HH
#define TOKENIZE_HH
#include <string>
#include <vector>
static std::vector< std::string > split( const std::string & str, const std::string & separator )
{
std::vector< size_t > indices;
size_t next_token = 0;
while ( (next_token = str.find( separator, next_token )) != std::string::npos ) {
indices.push_back( next_token );
next_token++;
}
if ( indices.empty() ) {
return { str };
}
std::vector< std::string > ret;
/* first token */
ret.push_back( str.substr( 0, indices[ 0 ] ) );
/* inner tokens */
for ( size_t i = 0; i < indices.size() - 1; i++ ) {
ret.push_back( str.substr( indices[ i ] + separator.size(),
indices[ i + 1 ] - indices[ i ] - separator.size() ) );
}
/* last token */
ret.push_back( str.substr( indices.back() + separator.size() ) );
return ret;
}
#endif /* TOKENIZE_HH */
src/httpserver/Makefile.am
+8
View File
@@ -0,0 +1,8 @@
AM_CPPFLAGS = -I$(srcdir)/../util -I$(srcdir)/../http -I../protobufs $(CXX11_FLAGS)
AM_CXXFLAGS = $(PICKY_CXXFLAGS)
noinst_LIBRARIES = libhttpserver.a
libhttpserver_a_SOURCES = http_proxy.hh http_proxy.cc \
secure_socket.hh secure_socket.cc certificate.hh \
apache_configuration.hh
src/httpserver/apache_configuration.hh
+14
View File
@@ -0,0 +1,14 @@
/* -*-mode:c++; tab-width: 4; indent-tabs-mode: nil; c-basic-offset: 4 -*- */
#include <string>
#include "config.h"
#ifndef APACHE_CONFIGURATION_HH
#define APACHE_CONFIGURATION_HH
const std::string apache_main_config = "LoadModule mpm_prefork_module " + std::string( MOD_MPM_PREFORK ) + "\nLoadModule authz_core_module " + std::string( MOD_AUTHZ_CORE ) + "\nMutex pthread\nLoadFile " + std::string( MOD_DEEPCGI ) + "\nLoadModule deepcgi_module " + std::string( MOD_DEEPCGI ) + "\nSetHandler deepcgi-handler\n";
const std::string apache_ssl_config = "LoadModule ssl_module " + std::string( MOD_SSL ) + "\nSSLEngine on\nSSLCertificateFile " + std::string( MOD_SSL_CERTIFICATE_FILE ) + "\nSSLCertificateKeyFile " + std::string( MOD_SSL_KEY ) +"\n";
#endif /* APACHE_CONFIGURATION_HH */
src/httpserver/certificate.hh
+168
View File
@@ -0,0 +1,168 @@
/* -*-mode:c++; tab-width: 4; indent-tabs-mode: nil; c-basic-offset: 4 -*- */
#ifndef CERTIFICATE_HH
#define CERTIFICATE_HH
unsigned char private_key[] = {
0x30, 0x82, 0x04, 0xa3, 0x02, 0x01, 0x00, 0x02, 0x82, 0x01, 0x01, 0x00,
0xc2, 0x70, 0x86, 0x11, 0x74, 0xf8, 0x7e, 0xbf, 0xe6, 0x17, 0xdc, 0xa8,
0xc4, 0xa8, 0x22, 0xa1, 0x4f, 0x26, 0x75, 0x39, 0x01, 0x7a, 0x94, 0xe0,
0x1b, 0x12, 0xe9, 0xac, 0x1b, 0xdd, 0x8e, 0x03, 0x51, 0xdd, 0xe5, 0xb0,
0x4b, 0xae, 0x33, 0x37, 0x16, 0x0b, 0x01, 0xa7, 0x41, 0x8e, 0x81, 0xdc,
0xb2, 0xa5, 0xb6, 0xc4, 0xba, 0xf4, 0xf7, 0x71, 0xad, 0x79, 0x76, 0x8c,
0x88, 0x96, 0x74, 0xfa, 0xcb, 0x41, 0x4d, 0x4e, 0xb0, 0xd7, 0x9b, 0xe4,
0x9b, 0xef, 0x7a, 0xe1, 0x63, 0xb0, 0x12, 0xa1, 0x50, 0x6d, 0xb7, 0x9a,
0xc1, 0x22, 0x48, 0x4f, 0x7a, 0xd9, 0xc5, 0x1a, 0x4d, 0x85, 0xde, 0x00,
0x1f, 0xac, 0x02, 0xc4, 0x29, 0xc9, 0x20, 0x0f, 0xa5, 0x60, 0xa6, 0xbf,
0x5b, 0x16, 0xf2, 0x7f, 0xfe, 0x26, 0x28, 0xb6, 0xb6, 0x03, 0x77, 0xda,
0x13, 0xe7, 0x8f, 0xf1, 0x6b, 0x6d, 0xa2, 0x45, 0x40, 0x7a, 0xf7, 0xce,
0x83, 0x68, 0x27, 0xdf, 0x82, 0x18, 0x5d, 0x5b, 0xdd, 0xb2, 0x39, 0xf3,
0x85, 0xd0, 0xdc, 0x68, 0x03, 0x62, 0x76, 0x99, 0xa9, 0x6e, 0xba, 0x22,
0x66, 0xb8, 0xe1, 0x6d, 0x48, 0xf4, 0x4e, 0x6a, 0x93, 0x6c, 0xf6, 0x9c,
0xd9, 0xb3, 0x30, 0x82, 0x7d, 0xf8, 0xfc, 0xd2, 0x43, 0x70, 0x88, 0xb1,
0x1f, 0x6f, 0x38, 0xe9, 0x22, 0x04, 0x36, 0xb5, 0xe6, 0x4c, 0xae, 0xfc,
0x6d, 0x7c, 0x87, 0x09, 0x1f, 0xb4, 0x5c, 0xdd, 0x1b, 0x56, 0xfb, 0xbb,
0x33, 0x3a, 0x34, 0xa2, 0xdd, 0xad, 0x6c, 0x7f, 0x7b, 0xb1, 0x91, 0xe8,
0x4a, 0x8e, 0xe1, 0x11, 0x8c, 0x95, 0x1f, 0x8f, 0x17, 0x75, 0xbd, 0xd6,
0x54, 0x34, 0xb3, 0xff, 0xd9, 0x05, 0xd8, 0xa0, 0xaa, 0xb9, 0x04, 0x57,
0x3e, 0x3a, 0xeb, 0x87, 0xef, 0xe4, 0x8f, 0xfb, 0x2f, 0xbf, 0xb3, 0x41,
0x31, 0x09, 0x08, 0x43, 0x02, 0x03, 0x01, 0x00, 0x01, 0x02, 0x82, 0x01,
0x00, 0x41, 0x44, 0xa2, 0x4c, 0x53, 0xd6, 0x62, 0x3d, 0xaa, 0x50, 0x12,
0x3a, 0xf8, 0xa0, 0x73, 0xd7, 0x70, 0x09, 0x39, 0xca, 0x23, 0xeb, 0xf0,
0xf3, 0x75, 0x43, 0x18, 0xfb, 0x33, 0xf3, 0x60, 0x0b, 0x4f, 0x3c, 0xcc,
0x30, 0x2c, 0x87, 0x52, 0x1b, 0xc8, 0xa3, 0x73, 0x2f, 0xa0, 0xba, 0xe4,
0x6e, 0x9f, 0xed, 0x2e, 0x77, 0xc5, 0x46, 0x1b, 0x95, 0xa6, 0x24, 0xa2,
0x9d, 0xb3, 0xef, 0xc5, 0x5e, 0xb6, 0x41, 0xba, 0x60, 0x92, 0xce, 0x1d,
0x3f, 0x54, 0xf4, 0xc3, 0x6b, 0xcb, 0x19, 0xbc, 0x77, 0x37, 0xd7, 0x4c,
0x8d, 0x7f, 0x0d, 0x31, 0xf6, 0xb4, 0x07, 0x4b, 0xb9, 0x73, 0x92, 0x5f,
0x1d, 0xed, 0x83, 0xc0, 0xfd, 0xf6, 0x04, 0xd4, 0x65, 0x4c, 0x1f, 0x30,
0x7a, 0x05, 0x6a, 0xa3, 0x86, 0x41, 0x5a, 0xaf, 0xc9, 0xfd, 0x39, 0x16,
0xcf, 0x36, 0x65, 0x41, 0x95, 0x49, 0x79, 0x68, 0xc1, 0x7e, 0x74, 0xab,
0xb5, 0x89, 0xac, 0xe9, 0x54, 0xca, 0xa8, 0x69, 0xb9, 0x03, 0x3e, 0xe6,
0x2b, 0x7f, 0x1e, 0x42, 0xd0, 0x9a, 0x45, 0xe4, 0x5a, 0x15, 0xcd, 0x72,
0x92, 0x8e, 0x18, 0x60, 0x20, 0xe9, 0xfd, 0x48, 0x27, 0x90, 0x30, 0x83,
0x10, 0xa0, 0xda, 0x3f, 0x02, 0xda, 0x1e, 0x76, 0xe4, 0x89, 0x59, 0xe9,
0xb7, 0x8d, 0x87, 0x17, 0xa9, 0x0d, 0x5e, 0x87, 0x38, 0x38, 0x81, 0x3e,
0x58, 0x08, 0x15, 0x1d, 0x43, 0xce, 0x4c, 0x26, 0x09, 0xeb, 0xd4, 0xef,
0x79, 0x38, 0x64, 0xe3, 0x0c, 0xa3, 0x6d, 0x12, 0x6e, 0x40, 0xc2, 0x64,
0xb2, 0x40, 0xc4, 0xb7, 0x3b, 0x9b, 0xb5, 0x75, 0x67, 0x45, 0x28, 0xc9,
0xa8, 0x62, 0x0f, 0x25, 0x24, 0x04, 0x75, 0xd8, 0xad, 0xca, 0xf4, 0x31,
0xc2, 0xfd, 0x81, 0xfb, 0x5f, 0x7f, 0x87, 0x5b, 0x10, 0xe9, 0xe5, 0xe3,
0x71, 0x7d, 0x73, 0x31, 0x79, 0x02, 0x81, 0x81, 0x00, 0xe5, 0xe2, 0x52,
0xe0, 0x66, 0xca, 0x98, 0xc0, 0xa1, 0x49, 0x38, 0x5d, 0x9d, 0x32, 0xed,
0x93, 0x4c, 0x2a, 0xe9, 0x5c, 0x25, 0xfd, 0xa5, 0x3f, 0x23, 0x96, 0x2c,
0x68, 0x90, 0xae, 0xe5, 0xdc, 0x3d, 0x3a, 0x70, 0xba, 0x2c, 0x15, 0x82,
0x0f, 0xe1, 0x6c, 0xf0, 0x04, 0xaa, 0x65, 0xaf, 0x69, 0x7c, 0x8e, 0xfe,
0xa4, 0xd2, 0x1e, 0x52, 0xae, 0xbe, 0xcc, 0xac, 0xed, 0xba, 0xdd, 0x1d,
0x78, 0x91, 0xdb, 0x78, 0x56, 0xc0, 0x5d, 0x33, 0x10, 0xc2, 0xe3, 0x43,
0xfa, 0xe9, 0x54, 0x75, 0xd9, 0xbb, 0x34, 0x9d, 0xf7, 0x51, 0x14, 0xf2,
0x7c, 0x08, 0x82, 0x5d, 0x6e, 0xdd, 0x86, 0xdd, 0xf5, 0x85, 0xd9, 0xe9,
0xd9, 0x3e, 0x2c, 0x0b, 0x2b, 0xa2, 0xfc, 0x45, 0x3b, 0x49, 0xa8, 0x32,
0xaf, 0xd6, 0x61, 0x01, 0x56, 0x0a, 0x4b, 0x44, 0x29, 0x76, 0x5e, 0x7f,
0x52, 0xe8, 0x4b, 0x53, 0x7d, 0x02, 0x81, 0x81, 0x00, 0xd8, 0x87, 0x5f,
0x89, 0x72, 0xbc, 0x21, 0x0c, 0xfb, 0x12, 0x3d, 0x07, 0xb0, 0x98, 0xb2,
0x64, 0x54, 0x64, 0x84, 0x14, 0x1a, 0x4b, 0xa9, 0xc6, 0xdc, 0x4a, 0x49,
0xca, 0x2a, 0xa0, 0xdd, 0x30, 0xe0, 0x8d, 0x07, 0x00, 0x79, 0x1f, 0x36,
0x12, 0x0f, 0xda, 0x2b, 0x88, 0x46, 0x5e, 0xa2, 0x38, 0x0c, 0xe5, 0x1c,
0x69, 0x56, 0x0c, 0x3d, 0x4e, 0x25, 0x8b, 0xda, 0x2f, 0x22, 0xd7, 0xa6,
0x8a, 0x2b, 0xfa, 0xec, 0xd2, 0x9c, 0x98, 0xfd, 0x40, 0x5a, 0xee, 0xae,
0x67, 0xa6, 0xe8, 0xed, 0xb9, 0x09, 0x09, 0x7a, 0x70, 0x9f, 0x70, 0x4d,
0xdd, 0x30, 0x4a, 0x95, 0xda, 0xe6, 0xfe, 0x52, 0xf4, 0x21, 0xd5, 0xee,
0xe7, 0x1b, 0x76, 0x4c, 0xca, 0x76, 0xc9, 0x62, 0x92, 0x62, 0x1f, 0xf3,
0x42, 0x56, 0x0c, 0x0f, 0x40, 0xb9, 0x31, 0xb2, 0x97, 0xd7, 0xb8, 0x29,
0x50, 0x44, 0x9d, 0x16, 0xbf, 0x02, 0x81, 0x81, 0x00, 0xc1, 0x47, 0xff,
0x22, 0x89, 0x8a, 0x81, 0xd0, 0x44, 0x4b, 0x49, 0xc6, 0xfb, 0xea, 0xdb,
0xf1, 0x62, 0xb3, 0x19, 0xad, 0xce, 0x8b, 0x6a, 0xaa, 0xa1, 0x83, 0x1c,
0x27, 0x76, 0xae, 0x76, 0xb9, 0x2c, 0x09, 0xbc, 0xc9, 0xeb, 0x52, 0xd2,
0x1e, 0x04, 0x0d, 0x11, 0xe8, 0xfa, 0x53, 0xb0, 0x9e, 0x62, 0x89, 0xdf,
0xf4, 0x38, 0x6e, 0x22, 0x5a, 0xa6, 0x35, 0x74, 0x98, 0x14, 0xea, 0xe0,
0x62, 0x85, 0x50, 0x1f, 0xe7, 0xe0, 0xa8, 0xfb, 0xcc, 0x75, 0xef, 0x9b,
0x32, 0x95, 0x87, 0xdb, 0xe6, 0x8b, 0x0d, 0xfc, 0x5a, 0xfa, 0xa3, 0x40,
0x15, 0xa5, 0x19, 0xe6, 0xf0, 0xd5, 0xf5, 0x15, 0x1d, 0xdf, 0xc2, 0x8e,
0x69, 0x87, 0x6a, 0x84, 0xa1, 0xd5, 0xd1, 0x81, 0x6c, 0x0d, 0xcc, 0x0b,
0x81, 0xe8, 0x1f, 0x7b, 0x7e, 0x19, 0xfd, 0x9a, 0x7b, 0xb9, 0xf8, 0x4d,
0xd9, 0x29, 0x52, 0x5c, 0x09, 0x02, 0x81, 0x80, 0x1b, 0x25, 0xf6, 0x03,
0x8c, 0xc0, 0x2b, 0xc0, 0xad, 0xe6, 0x1e, 0x82, 0xcd, 0x50, 0x87, 0xe5,
0x6e, 0x73, 0x5f, 0xe0, 0x09, 0x5f, 0xfd, 0x3c, 0x72, 0xb8, 0x59, 0xc2,
0x20, 0x50, 0x25, 0x0b, 0x90, 0xcb, 0xfd, 0xd9, 0xc0, 0x55, 0xfa, 0x78,
0x8f, 0x46, 0x42, 0x79, 0x06, 0xa1, 0xac, 0xe3, 0x2c, 0x32, 0xb9, 0xba,
0x37, 0x50, 0xb7, 0x8b, 0x09, 0xc8, 0x1c, 0x26, 0x16, 0xcf, 0x45, 0x11,
0x43, 0xec, 0x65, 0x2f, 0x0e, 0xf5, 0x5b, 0xa3, 0x6b, 0x03, 0x4c, 0x18,
0xb0, 0xe3, 0xb9, 0x92, 0x9e, 0xa0, 0x72, 0x9a, 0x96, 0x53, 0x05, 0x60,
0x3b, 0x42, 0xce, 0xc1, 0x30, 0x7f, 0x28, 0xb5, 0x75, 0x5c, 0x96, 0x64,
0xa1, 0x9b, 0xd6, 0x8a, 0xf8, 0x13, 0xcd, 0xe9, 0x1e, 0xa6, 0x74, 0xe0,
0x97, 0x5b, 0x51, 0xba, 0x42, 0xbe, 0x09, 0x6f, 0x62, 0x5b, 0xb3, 0xca,
0x5d, 0x9f, 0xf8, 0x8f, 0x02, 0x81, 0x80, 0x04, 0x7a, 0x71, 0xec, 0x40,
0xe4, 0x53, 0xcc, 0x7f, 0xd1, 0x04, 0x91, 0x79, 0x2c, 0x84, 0x90, 0x93,
0x3d, 0x40, 0x28, 0xd6, 0x47, 0xab, 0xb4, 0xa6, 0x4e, 0xc7, 0xc4, 0x58,
0xcb, 0x7f, 0x9e, 0xd4, 0xe7, 0x78, 0x26, 0xa5, 0x61, 0x06, 0x2f, 0xf8,
0xcc, 0xe6, 0xd4, 0x80, 0xe5, 0x28, 0xf0, 0xfc, 0x94, 0x63, 0xd7, 0xcd,
0x0a, 0xc5, 0x87, 0x30, 0x4f, 0x58, 0x26, 0xbc, 0xe9, 0xb5, 0x61, 0x05,
0xe8, 0x1b, 0xc7, 0xa9, 0x2e, 0x31, 0x17, 0x81, 0xe0, 0xd6, 0x16, 0xe5,
0xf4, 0x97, 0xd2, 0x75, 0x1d, 0xaa, 0x35, 0x0f, 0x54, 0xee, 0x03, 0x4d,
0x8d, 0x8e, 0x83, 0x96, 0x44, 0xb8, 0x92, 0x0e, 0x27, 0xb0, 0x08, 0x42,
0x78, 0xff, 0xff, 0x03, 0x4a, 0x56, 0xc2, 0xba, 0xca, 0x93, 0x94, 0x23,
0x4d, 0x68, 0x9a, 0xbb, 0xf4, 0x02, 0x21, 0x2a, 0xcc, 0xac, 0x08, 0x43,
0xfb, 0xd0, 0x67 };
unsigned char certificate[] = {
0x30, 0x82, 0x02, 0xa2, 0x30, 0x82, 0x01, 0x8a, 0x02, 0x09, 0x00, 0x88,
0x3d, 0x73, 0xcd, 0x2d, 0x4d, 0x6e, 0x84, 0x30, 0x0d, 0x06, 0x09, 0x2a,
0x86, 0x48, 0x86, 0xf7, 0x0d, 0x01, 0x01, 0x05, 0x05, 0x00, 0x30, 0x13,
0x31, 0x11, 0x30, 0x0f, 0x06, 0x03, 0x55, 0x04, 0x03, 0x13, 0x08, 0x6d,
0x61, 0x68, 0x69, 0x6d, 0x61, 0x68, 0x69, 0x30, 0x1e, 0x17, 0x0d, 0x31,
0x33, 0x31, 0x32, 0x32, 0x31, 0x30, 0x31, 0x30, 0x34, 0x33, 0x37, 0x5a,
0x17, 0x0d, 0x31, 0x34, 0x30, 0x31, 0x32, 0x30, 0x30, 0x31, 0x30, 0x34,
0x33, 0x37, 0x5a, 0x30, 0x13, 0x31, 0x11, 0x30, 0x0f, 0x06, 0x03, 0x55,
0x04, 0x03, 0x13, 0x08, 0x6d, 0x61, 0x68, 0x69, 0x6d, 0x61, 0x68, 0x69,
0x30, 0x82, 0x01, 0x22, 0x30, 0x0d, 0x06, 0x09, 0x2a, 0x86, 0x48, 0x86,
0xf7, 0x0d, 0x01, 0x01, 0x01, 0x05, 0x00, 0x03, 0x82, 0x01, 0x0f, 0x00,
0x30, 0x82, 0x01, 0x0a, 0x02, 0x82, 0x01, 0x01, 0x00, 0xc2, 0x70, 0x86,
0x11, 0x74, 0xf8, 0x7e, 0xbf, 0xe6, 0x17, 0xdc, 0xa8, 0xc4, 0xa8, 0x22,
0xa1, 0x4f, 0x26, 0x75, 0x39, 0x01, 0x7a, 0x94, 0xe0, 0x1b, 0x12, 0xe9,
0xac, 0x1b, 0xdd, 0x8e, 0x03, 0x51, 0xdd, 0xe5, 0xb0, 0x4b, 0xae, 0x33,
0x37, 0x16, 0x0b, 0x01, 0xa7, 0x41, 0x8e, 0x81, 0xdc, 0xb2, 0xa5, 0xb6,
0xc4, 0xba, 0xf4, 0xf7, 0x71, 0xad, 0x79, 0x76, 0x8c, 0x88, 0x96, 0x74,
0xfa, 0xcb, 0x41, 0x4d, 0x4e, 0xb0, 0xd7, 0x9b, 0xe4, 0x9b, 0xef, 0x7a,
0xe1, 0x63, 0xb0, 0x12, 0xa1, 0x50, 0x6d, 0xb7, 0x9a, 0xc1, 0x22, 0x48,
0x4f, 0x7a, 0xd9, 0xc5, 0x1a, 0x4d, 0x85, 0xde, 0x00, 0x1f, 0xac, 0x02,
0xc4, 0x29, 0xc9, 0x20, 0x0f, 0xa5, 0x60, 0xa6, 0xbf, 0x5b, 0x16, 0xf2,
0x7f, 0xfe, 0x26, 0x28, 0xb6, 0xb6, 0x03, 0x77, 0xda, 0x13, 0xe7, 0x8f,
0xf1, 0x6b, 0x6d, 0xa2, 0x45, 0x40, 0x7a, 0xf7, 0xce, 0x83, 0x68, 0x27,
0xdf, 0x82, 0x18, 0x5d, 0x5b, 0xdd, 0xb2, 0x39, 0xf3, 0x85, 0xd0, 0xdc,
0x68, 0x03, 0x62, 0x76, 0x99, 0xa9, 0x6e, 0xba, 0x22, 0x66, 0xb8, 0xe1,
0x6d, 0x48, 0xf4, 0x4e, 0x6a, 0x93, 0x6c, 0xf6, 0x9c, 0xd9, 0xb3, 0x30,
0x82, 0x7d, 0xf8, 0xfc, 0xd2, 0x43, 0x70, 0x88, 0xb1, 0x1f, 0x6f, 0x38,
0xe9, 0x22, 0x04, 0x36, 0xb5, 0xe6, 0x4c, 0xae, 0xfc, 0x6d, 0x7c, 0x87,
0x09, 0x1f, 0xb4, 0x5c, 0xdd, 0x1b, 0x56, 0xfb, 0xbb, 0x33, 0x3a, 0x34,
0xa2, 0xdd, 0xad, 0x6c, 0x7f, 0x7b, 0xb1, 0x91, 0xe8, 0x4a, 0x8e, 0xe1,
0x11, 0x8c, 0x95, 0x1f, 0x8f, 0x17, 0x75, 0xbd, 0xd6, 0x54, 0x34, 0xb3,
0xff, 0xd9, 0x05, 0xd8, 0xa0, 0xaa, 0xb9, 0x04, 0x57, 0x3e, 0x3a, 0xeb,
0x87, 0xef, 0xe4, 0x8f, 0xfb, 0x2f, 0xbf, 0xb3, 0x41, 0x31, 0x09, 0x08,
0x43, 0x02, 0x03, 0x01, 0x00, 0x01, 0x30, 0x0d, 0x06, 0x09, 0x2a, 0x86,
0x48, 0x86, 0xf7, 0x0d, 0x01, 0x01, 0x05, 0x05, 0x00, 0x03, 0x82, 0x01,
0x01, 0x00, 0x80, 0xba, 0x06, 0xdc, 0x05, 0xc4, 0x5b, 0x99, 0x60, 0x63,
0x14, 0x2e, 0xc9, 0x9e, 0xb0, 0xd9, 0x85, 0xf2, 0xeb, 0x8e, 0x61, 0x56,
0xc3, 0xb9, 0x27, 0xfd, 0xf2, 0xfd, 0x2b, 0x9e, 0x5a, 0x7b, 0x82, 0x99,
0x51, 0x5d, 0xed, 0xe7, 0x01, 0xda, 0x15, 0x29, 0x13, 0x33, 0x0d, 0x81,
0xd1, 0x35, 0x3b, 0x21, 0x49, 0x37, 0x4f, 0x2a, 0x5f, 0x41, 0x2c, 0x57,
0x00, 0x06, 0x58, 0x11, 0x40, 0x56, 0x05, 0x23, 0x6f, 0x46, 0x2b, 0xea,
0x88, 0x56, 0x0f, 0x28, 0xae, 0xbc, 0x7a, 0x17, 0x39, 0x9f, 0xa9, 0x0d,
0xf0, 0x2b, 0x1b, 0x33, 0x8d, 0xc3, 0x13, 0x18, 0x51, 0x6d, 0x16, 0x7b,
0xfd, 0xa7, 0xd5, 0x33, 0x5a, 0x8e, 0x58, 0x1c, 0x92, 0xa5, 0x9d, 0x7e,
0x77, 0xee, 0xcb, 0x85, 0xa0, 0xde, 0x2e, 0x05, 0xb1, 0xbd, 0xbc, 0x12,
0xeb, 0x07, 0x0b, 0x0e, 0xd7, 0x4b, 0x71, 0x8d, 0x4a, 0x92, 0xa4, 0x2c,
0xd7, 0x2a, 0x0d, 0xaa, 0x19, 0xa1, 0x60, 0xfa, 0xd9, 0x57, 0xe5, 0xe9,
0x15, 0x20, 0x08, 0xfb, 0x02, 0x6d, 0xc6, 0x74, 0x08, 0xbf, 0x20, 0x33,
0xb2, 0x71, 0x22, 0x14, 0x56, 0xf7, 0xfd, 0x15, 0x78, 0x3d, 0x56, 0x64,
0x3e, 0x38, 0xdb, 0x90, 0xbd, 0x44, 0x00, 0xd7, 0x9a, 0x9c, 0x8b, 0xda,
0x8d, 0xd0, 0x47, 0x79, 0x98, 0x2f, 0xc5, 0xb2, 0x51, 0x4b, 0xef, 0x5a,
0x39, 0x40, 0xbf, 0x2c, 0x5b, 0x61, 0x9f, 0x18, 0xda, 0x96, 0x96, 0x88,
0x46, 0x7e, 0x35, 0x99, 0xe1, 0x37, 0x58, 0xc9, 0x8e, 0xf6, 0x64, 0x49,
0x54, 0x4d, 0x96, 0xfb, 0xc3, 0xa5, 0x2d, 0x7c, 0x5f, 0x60, 0x96, 0x8c,
0x8e, 0xa7, 0x3d, 0xd9, 0x4c, 0x97, 0x5a, 0xe1, 0x0d, 0x7b, 0xaf, 0x82,
0x20, 0xf8, 0xfc, 0x3c, 0x46, 0x61, 0x64, 0x77, 0xbd, 0xb5, 0xe9, 0x6f,
0x87, 0x65, 0x46, 0x54, 0xed, 0x5b
};
#endif /* CERTIFICATE_HH */
src/httpserver/http_proxy.cc
+139
View File
@@ -0,0 +1,139 @@
/* -*-mode:c++; tab-width: 4; indent-tabs-mode: nil; c-basic-offset: 4 -*- */
#include <thread>
#include <string>
#include <iostream>
#include <arpa/inet.h>
#include <linux/netfilter_ipv4.h>
#include "address.hh"
#include "socket.hh"
#include "system_runner.hh"
#include "http_proxy.hh"
#include "poller.hh"
#include "bytestream_queue.hh"
#include "http_request_parser.hh"
#include "http_response_parser.hh"
#include "file_descriptor.hh"
#include "event_loop.hh"
#include "temp_file.hh"
#include "secure_socket.hh"
#include "backing_store.hh"
#include "exception.hh"
using namespace std;
using namespace PollerShortNames;
HTTPProxy::HTTPProxy( const Address & listener_addr )
: listener_socket_(),
server_context_( SERVER ),
client_context_( CLIENT )
{
listener_socket_.bind( listener_addr );
listener_socket_.listen();
}
template <class SocketType>
void HTTPProxy::loop( SocketType & server, SocketType & client, HTTPBackingStore & backing_store )
{
Poller poller;
HTTPRequestParser request_parser;
HTTPResponseParser response_parser;
const Address server_addr = client.original_dest();
/* poll on original connect socket and new connection socket to ferry packets */
/* responses from server go to response parser */
poller.add_action( Poller::Action( server, Direction::In,
[&] () {
string buffer = server.read();
response_parser.parse( buffer );
return ResultType::Continue;
},
[&] () { return not client.eof(); } ) );
/* requests from client go to request parser */
poller.add_action( Poller::Action( client, Direction::In,
[&] () {
string buffer = client.read();
request_parser.parse( buffer );
return ResultType::Continue;
},
[&] () { return not server.eof(); } ) );
/* completed requests from client are serialized and sent to server */
poller.add_action( Poller::Action( server, Direction::Out,
[&] () {
server.write( request_parser.front().str() );
response_parser.new_request_arrived( request_parser.front() );
request_parser.pop();
return ResultType::Continue;
},
[&] () { return not request_parser.empty(); } ) );
/* completed responses from server are serialized and sent to client */
poller.add_action( Poller::Action( client, Direction::Out,
[&] () {
client.write( response_parser.front().str() );
backing_store.save( response_parser.front(), server_addr );
response_parser.pop();
return ResultType::Continue;
},
[&] () { return not response_parser.empty(); } ) );
while ( true ) {
if ( poller.poll( -1 ).result == Poller::Result::Type::Exit ) {
return;
}
}
}
void HTTPProxy::handle_tcp( HTTPBackingStore & backing_store )
{
thread newthread( [&] ( TCPSocket client ) {
/* get original destination for connection request */
Address server_addr = client.original_dest();
std::cout << "new TCPSocket original server addr: " << server_addr.str() << std::endl;
try {
/* create socket and connect to original destination and send original request */
TCPSocket server;
server.connect( server_addr );
if ( server_addr.port() == 80) { /* normal HTTP */
return loop( server, client, backing_store );
}
/* handle TLS */
SecureSocket tls_client( server_context_.new_secure_socket( move( client ) ) );
SecureSocket tls_server( client_context_.new_secure_socket( move( server ) ) );
tls_client.accept();
std::string servername;
if ( tls_client.get_sni_servername( servername ) ) {
tls_server.set_sni_servername_sent( servername );
}
std::cout << "get SNI" << server_addr.ip() << "\t" << servername << endl;
tls_server.connect();
loop( tls_server, tls_client, backing_store );
} catch ( const exception & e ) {
print_exception( e );
}
}, listener_socket_.accept() );
/* don't wait around for the reply */
newthread.detach();
}
/* register this HTTPProxy's TCP listener socket to handle events with
the given event_loop, saving request-response pairs to the given
backing_store (which is captured and must continue to persist) */
void HTTPProxy::register_handlers( EventLoop & event_loop, HTTPBackingStore & backing_store )
{
event_loop.add_simple_input_handler( tcp_listener(),
[&] () {
handle_tcp( backing_store );
return ResultType::Continue;
} );
}
src/httpserver/http_proxy.hh
+41
View File
@@ -0,0 +1,41 @@
/* -*-mode:c++; tab-width: 4; indent-tabs-mode: nil; c-basic-offset: 4 -*- */
#ifndef HTTP_PROXY_HH
#define HTTP_PROXY_HH
#include <string>
#include "socket.hh"
#include "secure_socket.hh"
#include "http_response.hh"
class HTTPBackingStore;
class EventLoop;
class Poller;
class HTTPRequestParser;
class HTTPResponseParser;
class HTTPProxy
{
private:
TCPSocket listener_socket_;
template <class SocketType>
void loop( SocketType & server, SocketType & client, HTTPBackingStore & backing_store );
SSLContext server_context_, client_context_;
public:
HTTPProxy( const Address & listener_addr );
TCPSocket & tcp_listener( void ) { return listener_socket_; }
void handle_tcp( HTTPBackingStore & backing_store );
/* register this HTTPProxy's TCP listener socket to handle events with
the given event_loop, saving request-response pairs to the given
backing_store (which is captured and must continue to persist) */
void register_handlers( EventLoop & event_loop, HTTPBackingStore & backing_store );
};
#endif /* HTTP_PROXY_HH */
src/httpserver/nginx_configuration.hh
+40
View File
@@ -0,0 +1,40 @@
/* -*-mode:c++; tab-width: 4; indent-tabs-mode: nil; c-basic-offset: 4 -*- */
#include <string>
#include "config.h"
#ifndef NGINX_CONFIGURATION_HH
#define NGINX_CONFIGURATION_HH
const std::string nginx_main_cfg = R"END(
load_module "/data/tools/nginx/modules/ngx_quic_module.so";
#user root;
worker_processes 1;
daemon on;
worker_rlimit_core 8000m;
working_directory /data/tools/nginx;
error_log logs/error.log debug;
#error_log logs/error.log;
#error_log logs/error.log notice;
#error_log logs/error.log info;
#error_log "pipe:rollback logs/error_log interval=1d baknum=7 maxsize=2G";
#pid logs/nginx.pid;
events {
worker_connections 1024;
accept_mutex off;
}
)END";
//pid-file: status/pid-file
const std::string nginx_ssl_config_certfile =std::string( MOD_SSL_CERTIFICATE_FILE );
const std::string nginx_ssl_config_keyfile =std::string( MOD_SSL_KEY );
#endif /* NGINX_CONFIGURATION_HH */
src/httpserver/secure_socket.cc
+262
View File
@@ -0,0 +1,262 @@
/* -*-mode:c++; tab-width: 4; indent-tabs-mode: nil; c-basic-offset: 4 -*- */
#include <cassert>
#include <vector>
#include <thread>
#include <mutex>
#include <unordered_map>
#include "secure_socket.hh"
#include "certificate.hh"
#include "exception.hh"
#include <unistd.h>
#include <signal.h>
using namespace std;
/* error category for OpenSSL */
class ssl_error_category : public error_category
{
public:
const char * name( void ) const noexcept override { return "SSL"; }
string message( const int ssl_error ) const noexcept override
{
return ERR_error_string( ssl_error, nullptr );
}
};
class ssl_error : public tagged_error
{
public:
ssl_error( const string & s_attempt,
const int error_code = ERR_get_error() )
: tagged_error( ssl_error_category(), s_attempt, error_code )
{}
};
class OpenSSL
{
private:
vector<mutex> locks_;
static void locking_function( int mode, int n, const char *, int )
{
if ( mode & CRYPTO_LOCK ) {
OpenSSL::global_context().locks_.at( n ).lock();
} else {
OpenSSL::global_context().locks_.at( n ).unlock();
}
}
static unsigned long id_function( void )
{
return pthread_self();
}
public:
OpenSSL()
: locks_( CRYPTO_num_locks() )
{
/* SSL initialization: Needs to be done exactly once */
/* load algorithms/ciphers */
SSL_library_init();
OpenSSL_add_all_algorithms();
/* load error messages */
SSL_load_error_strings();
/* set thread-safe callbacks */
CRYPTO_set_locking_callback( locking_function );
CRYPTO_set_id_callback( id_function );
}
static OpenSSL & global_context( void )
{
static OpenSSL os;
return os;
}
};
SSL_CTX * initialize_new_context( const SSL_MODE type )
{
OpenSSL::global_context();
SSL_CTX * ret = SSL_CTX_new( type == CLIENT ? SSLv23_client_method() : SSLv23_server_method() );
if ( not ret ) {
throw ssl_error( "SSL_CTL_new" );
}
return ret;
}
/* associates an SSL object to the SNI servername received; protected by the mutex */
static std::unordered_map<SSL *, std::string> ssl_to_servername_;
static std::mutex servername_mutex_;
static int ssl_servername_cb( SSL * ssl,
int * ad __attribute((unused)),
void * arg __attribute((unused)) )
{
const char * servername = SSL_get_servername( ssl, TLSEXT_NAMETYPE_host_name );
if ( servername && *servername ) {
std::lock_guard<std::mutex> lg( servername_mutex_ );
ssl_to_servername_[ ssl ] = servername;
}
return SSL_TLSEXT_ERR_OK;
}
SSLContext::SSLContext( const SSL_MODE type )
: ctx_( initialize_new_context( type ) )
{
if ( type == SERVER ) {
if ( not SSL_CTX_use_certificate_ASN1( ctx_.get(), 678, certificate ) ) {
throw ssl_error( "SSL_CTX_use_certificate_ASN1" );
}
if ( not SSL_CTX_use_RSAPrivateKey_ASN1( ctx_.get(), private_key, 1191 ) ) {
throw ssl_error( "SSL_CTX_use_RSAPrivateKey_ASN1" );
}
/* check consistency of private key with loaded certificate */
if ( not SSL_CTX_check_private_key( ctx_.get() ) ) {
throw ssl_error( "SSL_CTX_check_private_key" );
}
/* callback to get SNI information from client */
SSL_CTX_set_tlsext_servername_callback( ctx_.get(), ssl_servername_cb );
}
}
SecureSocket::SecureSocket( TCPSocket && sock, SSL * ssl )
: TCPSocket( move( sock ) ),
ssl_( ssl )
{
if ( not ssl_ ) {
throw runtime_error( "SecureSocket: constructor must be passed valid SSL structure" );
}
if ( not SSL_set_fd( ssl_.get(), fd_num() ) ) {
throw ssl_error( "SSL_set_fd" );
}
/* enable read/write to return only after handshake/renegotiation and successful completion */
SSL_set_mode( ssl_.get(), SSL_MODE_AUTO_RETRY );
}
SecureSocket SSLContext::new_secure_socket( TCPSocket && sock )
{
std::cout << sock.original_dest().str() << std::endl;
return SecureSocket( move( sock ),
SSL_new( ctx_.get() ) );
}
void SecureSocket::connect( void )
{
#if 0
if ( not SSL_connect( ssl_.get() ) ) {
throw ssl_error( "SSL_connect" );
}
#endif
#if 0
std::cout << "begin ssl connect()" << std::endl;
int ret = 0;
signal(SIGALRM, myalarm);
alarm(5);
ret = SSL_connect(ssl_.get());
std::cout << "SSL_connect: " << ret << std::endl;
if (ret < 0) {
return;
}
alarm(0);
#endif
int ret = SSL_connect(ssl_.get());
std::cout << "SSL_connect: " << ret << std::endl;
if (ret != 1) {
throw ssl_error("SSL_connect");
}
}
void SecureSocket::accept( void )
{
const auto ret = SSL_accept( ssl_.get() );
if ( ret == 1 ) {
return;
} else {
throw ssl_error( "SSL_accept" );
}
register_read();
}
string SecureSocket::read( void )
{
/* SSL record max size is 16kB */
const size_t SSL_max_record_length = 16384;
char buffer[ SSL_max_record_length ];
ssize_t bytes_read = SSL_read( ssl_.get(), buffer, SSL_max_record_length );
/* Make sure that we really are reading from the underlying fd */
assert( 0 == SSL_pending( ssl_.get() ) );
if ( bytes_read == 0 ) {
int error_return = SSL_get_error( ssl_.get(), bytes_read );
if ( SSL_ERROR_ZERO_RETURN == error_return ) { /* Clean SSL close */
set_eof();
} else if ( SSL_ERROR_SYSCALL == error_return ) { /* Underlying TCP connection close */
/* Verify error queue is empty so we can conclude it is EOF */
assert( ERR_get_error() == 0 );
set_eof();
}
register_read();
return string(); /* EOF */
} else if ( bytes_read < 0 ) {
throw ssl_error( "SSL_read" );
} else {
/* success */
register_read();
return string( buffer, bytes_read );
}
}
void SecureSocket::write(const string & message )
{
/* SSL_write returns with success if complete contents of message are written */
ssize_t bytes_written = SSL_write( ssl_.get(), message.data(), message.length() );
if ( bytes_written < 0 ) {
throw ssl_error( "SSL_write" );
}
register_write();
}
bool SecureSocket::get_sni_servername( std::string & servername )
{
std::lock_guard<std::mutex> lg( servername_mutex_ );
auto it = ssl_to_servername_.find( ssl_.get() );
if ( it == ssl_to_servername_.end() ) {
return false;
}
servername = it->second;
return true;
}
void SecureSocket::set_sni_servername_sent( const std::string & servername )
{
if ( !SSL_set_tlsext_host_name( ssl_.get(), servername.c_str() ) ) {
throw ssl_error( "SSL_set_tlsext_host_name" );
}
}
src/httpserver/secure_socket.hh
+54
View File
@@ -0,0 +1,54 @@
/* -*-mode:c++; tab-width: 4; indent-tabs-mode: nil; c-basic-offset: 4 -*- */
/* run: sudo apt-get install libssl-dev */
#ifndef SECURE_SOCKET_HH
#define SECURE_SOCKET_HH
#include <openssl/ssl.h>
#include <openssl/err.h>
#include "socket.hh"
enum SSL_MODE { CLIENT, SERVER };
class SecureSocket : public TCPSocket
{
friend class SSLContext;
private:
struct SSL_deleter { void operator()( SSL * x ) const { SSL_free( x ); } };
typedef std::unique_ptr<SSL, SSL_deleter> SSL_handle;
SSL_handle ssl_;
SecureSocket( TCPSocket && sock, SSL * ssl );
public:
void connect( void );
void accept( void );
std::string read( void );
void write( const std::string & message );
/* if an SNI servername has been received, assigns it to servername and returns true */
bool get_sni_servername( std::string & servername );
void set_sni_servername_sent( const std::string & servername );
};
class SSLContext
{
private:
struct CTX_deleter { void operator()( SSL_CTX * x ) const { SSL_CTX_free( x ); } };
typedef std::unique_ptr<SSL_CTX, CTX_deleter> CTX_handle;
CTX_handle ctx_;
static void set_servername( SSL * ssl, const std::string & servername );
static bool get_servername( SSL * ssl, std::string & servername );
public:
SSLContext( const SSL_MODE type );
SecureSocket new_secure_socket( TCPSocket && sock );
};
#endif
src/packet/Makefile.am
Executable
+11
View File
@@ -0,0 +1,11 @@
AM_CPPFLAGS = -I$(srcdir)/../util $(CXX11_FLAGS)
AM_CXXFLAGS = $(PICKY_CXXFLAGS)
noinst_LIBRARIES = libpacket.a
libpacket_a_SOURCES = packetshell.hh packetshell.cc queued_packet.hh \
abstract_packet_queue.hh dropping_packet_queue.hh dropping_packet_queue.cc infinite_packet_queue.hh \
drop_tail_packet_queue.hh drop_head_packet_queue.hh \
codel_packet_queue.cc codel_packet_queue.hh \
pie_packet_queue.cc pie_packet_queue.hh \
bindworkaround.hh
Loaded 100 of 180 files, more files were not shown because too many files have changed in this diff. Show more