32 Commits

Author SHA1 Message Date
Junxiao Shi 7639daa564 consumer: gather constructor args into Options struct
refs #5069

Change-Id: I4eeacc2045157dd12728dab589538d6994ffcb5a
2023-11-26 02:16:21 +00:00
Davide Pesavento 60e5043d0c Switch to Face::getIoContext()
Change-Id: I56ea53ce8db3a3f24abe958065b9251eb025fbf7
2023-11-11 18:35:05 -05:00
Davide Pesavento 690272d427 build: require boost >= 1.71.0
Refs: #5276
Change-Id: Ia1014604d6933ca0042cecbaa25e8df2e7e2356c
2023-09-23 20:33:42 -04:00
Davide Pesavento 0b272e0e69 build: more accurate dependencies on Boost libraries
See also named-data/ndn-cxx@5686c51b87

Change-Id: I0ff9b60ae9a5e51f597d16221bbe1d97afc9704a
2023-09-23 20:22:01 -04:00
Davide Pesavento 9dea7a2334 build: update waf to version 2.0.26
Change-Id: I3e8d2fe200ae8502bbc3ce423a611a0d95245c83
2023-08-19 02:04:03 -04:00
Davide Pesavento 822c579cb6 build: sync default build flags with ndn-cxx
Change-Id: I74812a748fd65151a462c3a1f0beb8dcf3241c28
2023-08-16 15:06:23 -04:00
Junxiao Shi c5f5eb1f14 Change namespace of DummyClientFace, Signal, Segmenter
refs #3940

Change-Id: Id014e1487bfaceac96e97af51dcfd77c4964ef11
2023-08-11 08:05:23 +00:00
Davide Pesavento 88b7bbd797 build: align minimum build requirements with ndn-cxx
Change-Id: I717d0a2ade24bfd6884f54209f06944601e8fe1a
2023-04-26 15:48:02 -04:00
Davide Pesavento 46bac78888 ci: adopt reusable workflows from named-data/actions
Switch source archive (tarball) to xz compression and
mention using gerrit for code review in the README

Change-Id: Ie69de065311737f6fe48976fbab5f896f3960e06
2023-02-23 20:01:25 -05:00
Davide Pesavento ec8f854672 Prepare release 0.4.0
Change-Id: I361b0a37f48c729b31ea6a61aefb513c8d3d670f
2023-01-18 23:10:24 -05:00
Davide Pesavento 36651f17f3 docs: improve formatting and document structure
Change-Id: I4b0b4cc3eae000aa1c988c5d308c3f78052ad96d
2023-01-05 04:07:05 -05:00
Davide Pesavento 59f6fabe50 build: update waf to version 2.0.24
Change-Id: I3b7226af85fa686d33866899a9c2e487313fc842
2022-12-02 20:36:32 -05:00
Saurab Dulal 737f2de0bd partial-sync: add unsubscribe feature
Change-Id: Ib32aa2c0acebb5be40bb8d99689a3ea18113a692
2022-11-22 04:53:29 +00:00
Davide Pesavento b68f28497e Use ndn-cxx's Segmenter in SegmentPublisher
This commit also makes lld the preferred linker on Linux

Change-Id: I3a66f2e79a5378d005046c61a8ec2d49345508d6
2022-11-17 19:07:04 -05:00
Davide Pesavento 03426efa2c Minor code cleanups
Change-Id: I2b67216754ba30563a65a2697e6801e363986da1
2022-09-25 14:06:26 -04:00
Davide Pesavento c45a4eaef4 **breaking** Add KeyChain parameter to the producer API
Applications must be able to specify the KeyChain instance to use

Change-Id: If5054dbb8c00e2d18af8bf6a9199ded7266688f3
2022-09-25 14:04:36 -04:00
Davide Pesavento e010c78946 docs: simplify conf.py and fix compatibility with upcoming Sphinx 6.0
Change-Id: Id1ba3c27d4b1d295eb334f875931b56c0adcfc1b
2022-09-25 03:12:53 -04:00
Davide Pesavento da1a4d36a0 build+ci: support macOS on arm64
This commit also syncs the CI config and scripts with ndn-cxx

Change-Id: I0403bfcf211e41e15fa59bbbee4cbd1966d9c708
2022-08-19 19:54:33 -04:00
Davide Pesavento c407dee7bd build: migrate to C++17
Change-Id: Ic66b1703fc7d1d51d61c442de859e0921c0bdd7a
2022-07-27 01:50:24 -04:00
Davide Pesavento f278438bc7 build: support CentOS Stream 9
Change-Id: I8864a856b4f777110cea37528d963e9eb77ed973
2022-07-09 19:58:53 -04:00
Alexander Afanasyev 16d493eceb Add advisory "incomingFaceId" to missing data notifications
Change-Id: Ida1acdc7c86d1110a8586d4a396301b89c37229b
Refs: #3626
2022-05-19 12:22:36 -04:00
Davide Pesavento 0ba5eb6d6c ci: require gcovr 5.1 for code coverage reports
Stop passing `-j`, it wasn't very useful and it's broken in gcovr 5.1

Change-Id: Ie567fc8cf6ed70783c8f4f46983fe0d3a61e3d71
2022-04-19 18:05:16 -04:00
Davide Pesavento f6fd2fbd34 Convert to span and avoid deprecated ndn-cxx functions
Change-Id: I260639ef4c13e6a492b29d3b976cca0ae29876cb
2022-03-18 21:00:36 -04:00
Davide Pesavento 85a73d2b75 build: align minimum build dependencies with ndn-cxx
* gcc 7.4
 * clang 6.0
 * Xcode 10.0 (11.3 or later recommended)

Change-Id: I365e7b1e2c9b081e512f511bf5beee2487cfc361
2022-03-16 20:08:16 -07:00
Junxiao Shi fadcde5921 full-sync: cleanup test cases
refs #4659

Change-Id: I5043c1ed7698b7cdf6fc3774f5bfa4b0c05ea204
2022-02-28 00:38:45 +00:00
Junxiao Shi 32ccfc4d1d **breaking** use Name TLV-VALUE as input to murmurHash3
Previously the library was using Name URI string, which is not guaranteed
to be stable and unique.

refs #4838

Change-Id: Ic5e29f11f1320f95a754fb5def03faeb4d89cc01
2022-01-09 21:26:22 +00:00
Davide Pesavento e51936f8c6 build: update waf to version 2.0.23
Change-Id: I259ccd62d04ced56eb8dedab612de02789ba2971
2022-01-02 18:44:21 -05:00
Davide Pesavento f078fb5134 ci: add code coverage reporting via gcovr
Change-Id: I246bb21b55c40182184f849dedd27be81be202b1
2021-11-05 16:08:44 -04:00
Davide Pesavento 51189595bd docs: various fixes and cleanups
Change-Id: Ib20f89395bf7864e04080c0e69aec93ab8099a47
2021-10-03 17:40:04 -04:00
Davide Pesavento 8dce194d9a ci: replace Travis with GitHub Actions
Change-Id: Id9f6bc462115c1c49f02b8b75d65304f777e0ffd
2021-10-02 22:31:26 -04:00
Davide Pesavento 9f14dcd319 ci: don't use deprecated ndnsec commands
Change-Id: Ie85898f91fa44a22d87bb79aec7ebe4a7e5ef42e
2021-06-04 13:41:43 -04:00
Davide Pesavento 75496288d0 build: make graphviz optional for building documentation
Change-Id: I779e69037ecef1d49e035a572a843ef3a861e614
2021-04-26 00:06:26 -04:00
72 changed files with 1502 additions and 1512 deletions
+12
View File
@@ -0,0 +1,12 @@
name: CI
on:
push:
workflow_dispatch:
permissions: {}
jobs:
Ubuntu:
uses: named-data/actions/.github/workflows/jenkins-script-ubuntu.yml@v1
macOS:
uses: named-data/actions/.github/workflows/jenkins-script-macos.yml@v1
+19
View File
@@ -0,0 +1,19 @@
name: Docs
on:
push:
workflow_dispatch:
inputs:
skip-deploy:
description: 'Skip deployment?'
required: true
type: boolean
permissions: {}
jobs:
html:
uses: named-data/actions/.github/workflows/docs-html.yml@v1
with:
# Deploy only the master branch and release tags
deploy: ${{ !inputs.skip-deploy && (github.ref == 'refs/heads/master' || startsWith(github.ref, 'refs/tags/0.')) }}
secrets: inherit
+36 -16
View File
@@ -1,32 +1,52 @@
#!/usr/bin/env bash
set -e
source .jenkins.d/util.sh
set -eo pipefail
case $(uname) in
Linux)
if [[ -e /etc/os-release ]]; then
source /etc/os-release
else
source /usr/lib/os-release
fi
export ID VERSION_ID
export ID_LIKE="${ID} ${ID_LIKE} linux"
export PATH="${HOME}/.local/bin${PATH:+:}${PATH}"
;;
Darwin)
# Emulate a subset of os-release(5)
export ID=macos
export VERSION_ID=$(sw_vers -productVersion)
export PATH="/usr/local/bin${PATH:+:}${PATH}"
if [[ -x /opt/homebrew/bin/brew ]]; then
eval "$(/opt/homebrew/bin/brew shellenv)"
elif [[ -x /usr/local/bin/brew ]]; then
eval "$(/usr/local/bin/brew shellenv)"
fi
;;
esac
export CACHE_DIR=${CACHE_DIR:-/tmp}
export WAF_JOBS=${WAF_JOBS:-1}
[[ $JOB_NAME == *"code-coverage" ]] && export DISABLE_ASAN=yes
nanos() {
# Cannot use date(1) because macOS does not support %N format specifier
python3 -c 'import time; print(int(time.time() * 1e9))'
}
if [[ $JOB_NAME == *"code-coverage" ]]; then
export DISABLE_ASAN=yes
export DISABLE_HEADERS_CHECK=yes
fi
# https://reproducible-builds.org/docs/source-date-epoch/
export SOURCE_DATE_EPOCH=$(git log -1 --format=%ct)
for file in .jenkins.d/*; do
[[ -f $file && -x $file ]] || continue
if [[ -n $TRAVIS ]]; then
if [[ -n $GITHUB_ACTIONS ]]; then
label=$(basename "$file" | sed -E 's/[[:digit:]]+-(.*)\..*/\1/')
echo -ne "travis_fold:start:${label}\r"
echo -ne "travis_time:start:${label}\r"
start=$(nanos)
echo "::group::${label}"
fi
echo "\$ $file"
"$file"
if [[ -n $TRAVIS ]]; then
finish=$(nanos)
echo -ne "travis_time:end:${label}:start=${start},finish=${finish},duration=$((finish-start)),event=${label}\r"
echo -ne "travis_fold:end:${label}\r"
if [[ -n $GITHUB_ACTIONS ]]; then
echo "::endgroup::"
fi
done
+33 -29
View File
@@ -1,34 +1,38 @@
#!/usr/bin/env bash
set -ex
set -eo pipefail
if has OSX $NODE_LABELS; then
FORMULAE=(boost openssl pkg-config)
if has OSX-10.13 $NODE_LABELS || has OSX-10.14 $NODE_LABELS; then
FORMULAE+=(python)
APT_PKGS=(build-essential pkg-config python3-minimal
libboost-all-dev libssl-dev libsqlite3-dev)
FORMULAE=(boost openssl pkg-config)
PIP_PKGS=()
case $JOB_NAME in
*code-coverage)
APT_PKGS+=(lcov python3-pip)
PIP_PKGS+=('gcovr~=5.2')
;;
*Docs)
APT_PKGS+=(doxygen graphviz python3-pip)
FORMULAE+=(doxygen graphviz)
PIP_PKGS+=(sphinx sphinxcontrib-doxylink)
;;
esac
set -x
if [[ $ID == macos ]]; then
if [[ -n $GITHUB_ACTIONS ]]; then
export HOMEBREW_NO_INSTALLED_DEPENDENTS_CHECK=1
fi
if [[ -n $TRAVIS ]]; then
# Travis images come with a large number of pre-installed
# brew packages, don't waste time upgrading all of them
brew list --versions "${FORMULAE[@]}" || brew update
for FORMULA in "${FORMULAE[@]}"; do
brew list --versions "$FORMULA" || brew install "$FORMULA"
done
# Ensure /usr/local/opt/openssl exists
brew reinstall openssl
else
brew update
brew upgrade
brew install "${FORMULAE[@]}"
brew cleanup
fi
elif has Ubuntu $NODE_LABELS; then
brew update
brew install --formula "${FORMULAE[@]}"
elif [[ $ID_LIKE == *debian* ]]; then
sudo apt-get -qq update
sudo apt-get -qy install build-essential pkg-config python3-minimal \
libboost-all-dev libssl-dev libsqlite3-dev
if [[ $JOB_NAME == *"code-coverage" ]]; then
sudo apt-get -qy install gcovr lcov
fi
sudo apt-get -qy install "${APT_PKGS[@]}"
elif [[ $ID_LIKE == *fedora* ]]; then
sudo dnf -y install gcc-c++ libasan lld pkgconf-pkg-config python3 \
boost-devel openssl-devel sqlite-devel
fi
if (( ${#PIP_PKGS[@]} )); then
pip3 install --user --upgrade --upgrade-strategy=eager "${PIP_PKGS[@]}"
fi
+8 -13
View File
@@ -1,11 +1,11 @@
#!/usr/bin/env bash
set -ex
set -exo pipefail
pushd "$CACHE_DIR" >/dev/null
INSTALLED_VERSION=
if has OSX $NODE_LABELS; then
BOOST=$(brew ls --versions boost)
if [[ $ID == macos ]]; then
BOOST=$(brew list --formula --versions boost)
OLD_BOOST=$(cat boost.txt || :)
if [[ $OLD_BOOST != $BOOST ]]; then
echo "$BOOST" > boost.txt
@@ -35,21 +35,16 @@ sudo rm -f /usr/local/lib{,64}/pkgconfig/libndn-cxx.pc
pushd ndn-cxx >/dev/null
if has CentOS-8 $NODE_LABELS; then
# https://bugzilla.redhat.com/show_bug.cgi?id=1721553
PCH="--without-pch"
fi
./waf --color=yes configure --disable-static --enable-shared --without-osx-keychain $PCH
./waf --color=yes build -j$WAF_JOBS
sudo_preserve_env PATH -- ./waf --color=yes install
./waf --color=yes configure --without-osx-keychain
./waf --color=yes build
sudo ./waf --color=yes install
popd >/dev/null
popd >/dev/null
if has CentOS-8 $NODE_LABELS; then
if [[ $ID_LIKE == *fedora* ]]; then
sudo tee /etc/ld.so.conf.d/ndn.conf >/dev/null <<< /usr/local/lib64
fi
if has Linux $NODE_LABELS; then
if [[ $ID_LIKE == *linux* ]]; then
sudo ldconfig
fi
@@ -1,8 +1,8 @@
#!/usr/bin/env bash
set -ex
set -exo pipefail
PROJ=PSync
sudo rm -fr /usr/local/include/"$PROJ"
sudo rm -f /usr/local/lib{,64}/lib"$PROJ"*
sudo rm -f /usr/local/lib{,64}/pkgconfig/"$PROJ".pc
sudo rm -f /usr/local/lib{,64}/pkgconfig/{,lib}"$PROJ".pc
+15 -10
View File
@@ -1,8 +1,5 @@
#!/usr/bin/env bash
set -ex
git submodule sync
git submodule update --init
set -eo pipefail
if [[ -z $DISABLE_ASAN ]]; then
ASAN="--with-sanitizer=address"
@@ -10,32 +7,40 @@ fi
if [[ $JOB_NAME == *"code-coverage" ]]; then
COVERAGE="--with-coverage"
fi
if [[ $ID == debian && ${VERSION_ID%%.*} -eq 11 ]]; then
LZMA="--without-lzma"
fi
set -x
if [[ $JOB_NAME != *"code-coverage" && $JOB_NAME != *"limited-build" ]]; then
# Build in release mode with tests
./waf --color=yes configure --with-tests
./waf --color=yes build -j$WAF_JOBS
./waf --color=yes build
# Cleanup
./waf --color=yes distclean
# Build in release mode without tests
./waf --color=yes configure
./waf --color=yes build -j$WAF_JOBS
./waf --color=yes build
# Cleanup
./waf --color=yes distclean
fi
# Build in debug mode with tests and examples
./waf --color=yes configure --debug --with-tests --with-examples $ASAN $COVERAGE
./waf --color=yes build -j$WAF_JOBS
./waf --color=yes configure --debug --with-tests --with-examples $ASAN $COVERAGE $LZMA
./waf --color=yes build
# (tests will be run against the debug version)
# Install
sudo_preserve_env PATH -- ./waf --color=yes install
sudo ./waf --color=yes install
if has Linux $NODE_LABELS; then
if [[ $ID_LIKE == *fedora* ]]; then
sudo tee /etc/ld.so.conf.d/ndn.conf >/dev/null <<< /usr/local/lib64
fi
if [[ $ID_LIKE == *linux* ]]; then
sudo ldconfig
fi
+7 -5
View File
@@ -1,9 +1,5 @@
#!/usr/bin/env bash
set -ex
# Prepare environment
rm -rf ~/.ndn
ndnsec-keygen "/tmp/jenkins/$NODE_NAME" | ndnsec-install-cert -
set -eo pipefail
# https://github.com/google/sanitizers/wiki/AddressSanitizerFlags
ASAN_OPTIONS="color=always"
@@ -15,10 +11,16 @@ ASAN_OPTIONS+=":detect_invalid_pointer_pairs=2"
ASAN_OPTIONS+=":strip_path_prefix=${PWD}/"
export ASAN_OPTIONS
# https://www.boost.org/doc/libs/release/libs/test/doc/html/boost_test/runtime_config/summary.html
export BOOST_TEST_BUILD_INFO=1
export BOOST_TEST_COLOR_OUTPUT=1
export BOOST_TEST_DETECT_MEMORY_LEAK=0
export BOOST_TEST_LOGGER=HRF,test_suite,stdout:XML,all,build/xunit-log.xml
set -x
# Prepare environment
rm -rf ~/.ndn
# Run unit tests
./build/unit-tests
+14 -7
View File
@@ -1,23 +1,30 @@
#!/usr/bin/env bash
set -ex
set -exo pipefail
if [[ $JOB_NAME == *"code-coverage" ]]; then
# Generate an XML report (Cobertura format) and a detailed HTML report using gcovr
# Note: trailing slashes are important in the paths below. Do not remove them!
gcovr --object-directory build \
--filter PSync/ \
--exclude-throw-branches \
--exclude-unreachable-branches \
--cobertura build/coverage.xml \
--html-details build/gcovr/ \
--print-summary
# Generate a detailed HTML report using lcov
lcov --quiet \
--capture \
--directory . \
--exclude "$PWD/tests/*" \
--no-external \
--rc lcov_branch_coverage=1 \
--output-file build/coverage-with-tests.info
lcov --quiet \
--remove build/coverage-with-tests.info "$PWD/tests/*" \
--rc lcov_branch_coverage=1 \
--output-file build/coverage.info
genhtml --branch-coverage \
--demangle-cpp \
--legend \
--output-directory build/coverage \
--output-directory build/lcov \
--title "PSync unit tests" \
build/coverage.info
fi
+8 -10
View File
@@ -1,25 +1,24 @@
#!/usr/bin/env bash
set -eo pipefail
# It's intentional not to use `set -x`, because this script explicitly prints useful information
# and should not run in trace mode.
# It's intentional not to use `set -e`, because this script wants to check all headers
# (similar to running all test cases), instead of failing at the first error.
PROJ=PSync
PCFILE=PSync
if [[ -n $DISABLE_HEADERS_CHECK ]]; then
echo 'Skipping headers check.'
exit 0
fi
if has CentOS-8 $NODE_LABELS; then
if [[ $ID_LIKE == *fedora* ]]; then
export PKG_CONFIG_PATH=/usr/local/lib64/pkgconfig
fi
CXX=${CXX:-g++}
STD=-std=c++14
CXXFLAGS="-O2 -Wall -Wno-unneeded-internal-declaration -Wno-unused-const-variable $(pkg-config --cflags libndn-cxx $PROJ)"
INCLUDEDIR="$(pkg-config --variable=includedir $PROJ)"/$PROJ
STD=-std=c++17
CXXFLAGS="-O2 -Wall -Wno-unneeded-internal-declaration -Wno-unused-const-variable $(pkg-config --cflags libndn-cxx $PCFILE)"
INCLUDEDIR="$(pkg-config --variable=includedir $PCFILE)"/$PROJ
echo "Using: $CXX $STD $CXXFLAGS"
@@ -27,9 +26,8 @@ NCHECKED=0
NERRORS=0
while IFS= read -r -d '' H; do
echo "Checking header ${H#${INCLUDEDIR}/}"
"$CXX" -xc++ $STD $CXXFLAGS -c -o /dev/null "$H"
[[ $? -eq 0 ]] || ((NERRORS++))
((NCHECKED++))
"$CXX" -xc++ $STD $CXXFLAGS -c -o /dev/null "$H" || : $((NERRORS++))
: $((NCHECKED++))
done < <(find "$INCLUDEDIR" -name '*.hpp' -type f -print0 2>/dev/null)
if [[ $NCHECKED -eq 0 ]]; then
+24 -16
View File
@@ -1,28 +1,36 @@
# CONTINUOUS INTEGRATION SCRIPTS
# Continuous Integration Scripts
## Environment Variables Used in Build Scripts
## Environment Variables
- `NODE_LABELS`: space-separated list of platform properties. The included values are used by
the build scripts to select the proper behavior for different operating systems and versions.
- `ID`: lower-case string that identifies the operating system, for example: `ID=ubuntu`,
`ID=centos`. See [os-release(5)] for more information. On macOS, where `os-release` is
not available, we emulate it by setting `ID=macos`.
The list should normally contain `[OS_TYPE]`, `[DISTRO_TYPE]`, and `[DISTRO_VERSION]`.
- `ID_LIKE`: space-separated list of operating system identifiers that are closely related
to the running OS. See [os-release(5)] for more information. The listed values are used
by the CI scripts to select the proper behavior for different platforms and OS flavors.
Example values:
Examples:
- `[OS_TYPE]`: `Linux`, `OSX`
- `[DISTRO_TYPE]`: `Ubuntu`, `CentOS`
- `[DISTRO_VERSION]`: `Ubuntu-16.04`, `Ubuntu-18.04`, `CentOS-8`, `OSX-10.14`, `OSX-10.15`
- On CentOS, `ID_LIKE="centos rhel fedora linux"`
- On Ubuntu, `ID_LIKE="ubuntu debian linux"`
- `JOB_NAME`: optional variable that defines the type of build job. Depending on the job type,
the build scripts can perform different tasks.
- `VERSION_ID`: identifies the operating system version, excluding any release code names.
See [os-release(5)] for more information. Examples: `VERSION_ID=42`, `VERSION_ID=22.04`.
Possible values:
- `JOB_NAME`: defines the type of the current CI job. Depending on the job type, the CI
scripts can perform different tasks.
Supported values:
- empty: default build task
- `code-coverage`: debug build with tests and code coverage analysis (Ubuntu Linux is assumed)
- `code-coverage`: debug build with tests and code coverage analysis
- `limited-build`: only a single debug build with tests
- `CACHE_DIR`: directory containing cached files from previous builds, e.g., a compiled version
of ndn-cxx. If not set, `/tmp` is used.
- `CACHE_DIR`: directory containing cached files from previous builds, e.g., a compiled
version of ndn-cxx. If not set, `/tmp` is used.
- `WAF_JOBS`: number of parallel build threads used by waf, defaults to 1.
- `DISABLE_ASAN`: disable building with AddressSanitizer. This is automatically set for
the `code-coverage` job type.
[os-release(5)]: https://www.freedesktop.org/software/systemd/man/os-release.html
-39
View File
@@ -1,39 +0,0 @@
has() {
local saved_xtrace
[[ $- == *x* ]] && saved_xtrace=-x || saved_xtrace=+x
set +x
local p=$1
shift
local i ret=1
for i in "$@"; do
if [[ "${i}" == "${p}" ]]; then
ret=0
break
fi
done
set ${saved_xtrace}
return ${ret}
}
export -f has
sudo_preserve_env() {
local saved_xtrace
[[ $- == *x* ]] && saved_xtrace=-x || saved_xtrace=+x
set +x
local vars=()
while [[ $# -gt 0 ]]; do
local arg=$1
shift
case ${arg} in
--) break ;;
*) vars+=("${arg}=${!arg}") ;;
esac
done
set ${saved_xtrace}
sudo env "${vars[@]}" "$@"
}
export -f sudo_preserve_env
-106
View File
@@ -1,106 +0,0 @@
version: ~> 1.0
language: cpp
os: linux
dist: bionic
arch:
- amd64
- arm64
- ppc64le
- s390x
env:
- COMPILER=g++-7
- COMPILER=g++-9
- COMPILER=clang++-6.0
- COMPILER=clang++-9
jobs:
include:
# Linux
- env: COMPILER=g++-8
- env: COMPILER=clang++-5.0
- env: COMPILER=clang++-7
- env: COMPILER=clang++-8
- env: COMPILER=clang++-10
- env: COMPILER=clang++-11
- env: COMPILER=clang++-12
# macOS
- os: osx
osx_image: xcode9.4
env: # default compiler
- os: osx
osx_image: xcode10.1
env: # default compiler
- os: osx
osx_image: xcode10.3
env: # default compiler
- os: osx
osx_image: xcode11.3
env: # default compiler
- os: osx
osx_image: xcode11.6
env: # default compiler
- os: osx
osx_image: xcode12
env: # default compiler
allow_failures:
- arch: s390x # bug 4818
- env: COMPILER=clang++-12
fast_finish: true
before_install:
- |
: Adding apt repositories
case ${COMPILER} in
g++-9)
# https://launchpad.net/~ubuntu-toolchain-r/+archive/ubuntu/test/+packages
travis_retry sudo add-apt-repository -y ppa:ubuntu-toolchain-r/test
travis_retry sudo apt-get -qq update
;;
clang++-1?)
# https://apt.llvm.org/
LLVM_REPO=${COMPILER/clang++/llvm-toolchain-${TRAVIS_DIST}}
travis_retry wget -nv -O - "https://apt.llvm.org/llvm-snapshot.gpg.key" | sudo apt-key add -
travis_retry sudo add-apt-repository -y "deb http://apt.llvm.org/${TRAVIS_DIST}/ ${LLVM_REPO%-12} main"
travis_retry sudo apt-get -qq update
;;
esac
install:
- |
: Installing C++ compiler
if [[ -n ${COMPILER} ]]; then
travis_retry sudo apt-get -qy install ${COMPILER/clang++/clang}
fi
before_script:
- |
: Setting environment variables
if [[ -n ${COMPILER} ]]; then
export CXX=${COMPILER}
fi
case ${TRAVIS_OS_NAME} in
linux) export NODE_LABELS="Linux Ubuntu Ubuntu-18.04" ;;
osx) export NODE_LABELS="OSX OSX-$(sw_vers -productVersion | cut -d . -f -2)" ;;
esac
export WAF_JOBS=2
- |
: Enabling workarounds
case "${TRAVIS_CPU_ARCH},${COMPILER}" in
ppc64le,g++-7)
# AddressSanitizer does not seem to be working
export DISABLE_ASAN=yes
;;
*,clang++-8)
# https://bugs.llvm.org/show_bug.cgi?id=40808
export DISABLE_ASAN=yes
;;
esac
- ${CXX:-c++} --version
script:
- ./.jenkins
+2 -2
View File
@@ -54,8 +54,8 @@ from waflib import Utils, Logs, Errors
from waflib.Configure import conf
from waflib.TaskGen import feature, after_method
BOOST_LIBS = ['/usr/lib', '/usr/local/lib', '/opt/local/lib', '/sw/lib', '/lib']
BOOST_INCLUDES = ['/usr/include', '/usr/local/include', '/opt/local/include', '/sw/include']
BOOST_LIBS = ['/usr/lib', '/usr/local/lib', '/opt/homebrew/lib', '/opt/local/lib', '/sw/lib', '/lib']
BOOST_INCLUDES = ['/usr/include', '/usr/local/include', '/opt/homebrew/include', '/opt/local/include', '/sw/include']
BOOST_VERSION_FILE = 'boost/version.hpp'
BOOST_VERSION_CODE = '''
+92 -78
View File
@@ -3,10 +3,12 @@
import platform
from waflib import Configure, Logs, Utils
def options(opt):
opt.add_option('--debug', '--with-debug', action='store_true', default=False,
help='Compile in debugging mode with minimal optimizations (-Og)')
def configure(conf):
conf.start_msg('Checking C++ compiler version')
@@ -16,23 +18,27 @@ def configure(conf):
errmsg = ''
warnmsg = ''
if cxx == 'gcc':
if ccver < (5, 3, 0):
if ccver < (7, 4, 0):
errmsg = ('The version of gcc you are using is too old.\n'
'The minimum supported gcc version is 7.4.0.')
elif ccver < (7, 4, 0):
warnmsg = ('Using a version of gcc older than 7.4.0 is not '
'The minimum supported gcc version is 9.3.')
elif ccver < (9, 3, 0):
warnmsg = ('Using a version of gcc older than 9.3 is not '
'officially supported and may result in build failures.')
conf.flags = GccFlags()
elif cxx == 'clang':
if Utils.unversioned_sys_platform() == 'darwin' and ccver < (9, 0, 0):
errmsg = ('The version of Xcode you are using is too old.\n'
'The minimum supported Xcode version is 9.0.')
elif ccver < (4, 0, 0):
if Utils.unversioned_sys_platform() == 'darwin':
if ccver < (10, 0, 0):
errmsg = ('The version of Xcode you are using is too old.\n'
'The minimum supported Xcode version is 12.4.')
elif ccver < (12, 0, 0):
warnmsg = ('Using a version of Xcode older than 12.4 is not '
'officially supported and may result in build failures.')
elif ccver < (7, 0, 0):
errmsg = ('The version of clang you are using is too old.\n'
'The minimum supported clang version is 4.0.')
'The minimum supported clang version is 7.0.')
conf.flags = ClangFlags()
else:
warnmsg = '%s compiler is unsupported' % cxx
warnmsg = f'{cxx} compiler is unsupported'
conf.flags = CompilerFlags()
if errmsg:
@@ -44,7 +50,7 @@ def configure(conf):
else:
conf.end_msg(ccverstr)
conf.areCustomCxxflagsPresent = (len(conf.env.CXXFLAGS) > 0)
conf.areCustomCxxflagsPresent = len(conf.env.CXXFLAGS) > 0
# General flags are always applied (e.g., selecting C++ language standard)
generalFlags = conf.flags.getGeneralFlags(conf)
@@ -52,6 +58,7 @@ def configure(conf):
conf.add_supported_linkflags(generalFlags['LINKFLAGS'])
conf.env.DEFINES += generalFlags['DEFINES']
@Configure.conf
def check_compiler_flags(conf):
# Debug or optimized CXXFLAGS and LINKFLAGS are applied only if the
@@ -74,10 +81,11 @@ def check_compiler_flags(conf):
conf.env.DEFINES += extraFlags['DEFINES']
@Configure.conf
def add_supported_cxxflags(self, cxxflags):
"""
Check which cxxflags are supported by compiler and add them to env.CXXFLAGS variable
Check which cxxflags are supported by the active compiler and add them to env.CXXFLAGS variable.
"""
if len(cxxflags) == 0:
return
@@ -93,10 +101,11 @@ def add_supported_cxxflags(self, cxxflags):
self.end_msg(' '.join(supportedFlags))
self.env.prepend_value('CXXFLAGS', supportedFlags)
@Configure.conf
def add_supported_linkflags(self, linkflags):
"""
Check which linkflags are supported by compiler and add them to env.LINKFLAGS variable
Check which linkflags are supported by the active compiler and add them to env.LINKFLAGS variable.
"""
if len(linkflags) == 0:
return
@@ -113,13 +122,17 @@ def add_supported_linkflags(self, linkflags):
self.env.prepend_value('LINKFLAGS', supportedFlags)
class CompilerFlags(object):
class CompilerFlags:
def getCompilerVersion(self, conf):
return tuple(int(i) for i in conf.env.CC_VERSION)
def getGeneralFlags(self, conf):
"""Get dict of CXXFLAGS, LINKFLAGS, and DEFINES that are always needed"""
return {'CXXFLAGS': [], 'LINKFLAGS': [], 'DEFINES': []}
return {
'CXXFLAGS': [],
'LINKFLAGS': [],
'DEFINES': ['BOOST_ASIO_NO_DEPRECATED', 'BOOST_FILESYSTEM_NO_DEPRECATED'],
}
def getDebugFlags(self, conf):
"""Get dict of CXXFLAGS, LINKFLAGS, and DEFINES that are needed only in debug mode"""
@@ -129,99 +142,100 @@ class CompilerFlags(object):
"""Get dict of CXXFLAGS, LINKFLAGS, and DEFINES that are needed only in optimized mode"""
return {'CXXFLAGS': [], 'LINKFLAGS': [], 'DEFINES': ['NDEBUG']}
class GccBasicFlags(CompilerFlags):
class GccClangCommonFlags(CompilerFlags):
"""
This class defines basic flags that work for both gcc and clang compilers
This class defines common flags that work for both gcc and clang compilers.
"""
def getGeneralFlags(self, conf):
flags = super(GccBasicFlags, self).getGeneralFlags(conf)
flags['CXXFLAGS'] += ['-std=c++14']
if Utils.unversioned_sys_platform() == 'linux':
flags['LINKFLAGS'] += ['-fuse-ld=gold']
elif Utils.unversioned_sys_platform() == 'freebsd':
flags = super().getGeneralFlags(conf)
flags['CXXFLAGS'] += ['-std=c++17']
if Utils.unversioned_sys_platform() != 'darwin':
flags['LINKFLAGS'] += ['-fuse-ld=lld']
return flags
__cxxFlags = [
'-fdiagnostics-color',
'-Wall',
'-Wextra',
'-Wpedantic',
'-Wenum-conversion',
'-Wextra-semi',
'-Wnon-virtual-dtor',
'-Wno-unused-parameter',
]
__linkFlags = ['-Wl,-O1']
def getDebugFlags(self, conf):
flags = super(GccBasicFlags, self).getDebugFlags(conf)
flags['CXXFLAGS'] += ['-Og',
'-g3',
'-pedantic',
'-Wall',
'-Wextra',
'-Werror',
'-Wcatch-value=2',
'-Wextra-semi',
'-Wnon-virtual-dtor',
'-Wno-error=deprecated-declarations', # Bug #3795
'-Wno-error=maybe-uninitialized', # Bug #1615
'-Wno-unused-parameter',
]
flags['LINKFLAGS'] += ['-Wl,-O1']
flags = super().getDebugFlags(conf)
flags['CXXFLAGS'] += ['-Og', '-g'] + self.__cxxFlags + [
'-Werror',
'-Wno-error=deprecated-declarations', # Bug #3795
'-Wno-error=maybe-uninitialized', # Bug #1615
]
flags['LINKFLAGS'] += self.__linkFlags
return flags
def getOptimizedFlags(self, conf):
flags = super(GccBasicFlags, self).getOptimizedFlags(conf)
flags['CXXFLAGS'] += ['-O2',
'-g',
'-pedantic',
'-Wall',
'-Wextra',
'-Wcatch-value=2',
'-Wextra-semi',
'-Wnon-virtual-dtor',
'-Wno-unused-parameter',
]
flags['LINKFLAGS'] += ['-Wl,-O1']
flags = super().getOptimizedFlags(conf)
flags['CXXFLAGS'] += ['-O2', '-g1'] + self.__cxxFlags
flags['LINKFLAGS'] += self.__linkFlags
return flags
class GccFlags(GccBasicFlags):
class GccFlags(GccClangCommonFlags):
__cxxFlags = [
'-Wcatch-value=2',
'-Wcomma-subscript', # enabled by default in C++20
'-Wduplicated-branches',
'-Wduplicated-cond',
'-Wlogical-op',
'-Wredundant-tags',
'-Wvolatile', # enabled by default in C++20
]
def getDebugFlags(self, conf):
flags = super(GccFlags, self).getDebugFlags(conf)
flags['CXXFLAGS'] += ['-fdiagnostics-color',
'-Wredundant-tags',
]
if platform.machine() == 'armv7l' and self.getCompilerVersion(conf) >= (7, 1, 0):
flags = super().getDebugFlags(conf)
flags['CXXFLAGS'] += self.__cxxFlags
if platform.machine() == 'armv7l':
flags['CXXFLAGS'] += ['-Wno-psabi'] # Bug #5106
return flags
def getOptimizedFlags(self, conf):
flags = super(GccFlags, self).getOptimizedFlags(conf)
flags['CXXFLAGS'] += ['-fdiagnostics-color',
'-Wredundant-tags',
]
if platform.machine() == 'armv7l' and self.getCompilerVersion(conf) >= (7, 1, 0):
flags = super().getOptimizedFlags(conf)
flags['CXXFLAGS'] += self.__cxxFlags
if platform.machine() == 'armv7l':
flags['CXXFLAGS'] += ['-Wno-psabi'] # Bug #5106
return flags
class ClangFlags(GccBasicFlags):
class ClangFlags(GccClangCommonFlags):
def getGeneralFlags(self, conf):
flags = super(ClangFlags, self).getGeneralFlags(conf)
flags = super().getGeneralFlags(conf)
if Utils.unversioned_sys_platform() == 'darwin':
# Bug #4296
flags['CXXFLAGS'] += [['-isystem', '/usr/local/include'], # for Homebrew
['-isystem', '/opt/local/include']] # for MacPorts
brewdir = '/opt/homebrew' if platform.machine() == 'arm64' else '/usr/local'
flags['CXXFLAGS'] += [
['-isystem', f'{brewdir}/include'], # for Homebrew
['-isystem', '/opt/local/include'], # for MacPorts
]
elif Utils.unversioned_sys_platform() == 'freebsd':
# Bug #4790
flags['CXXFLAGS'] += [['-isystem', '/usr/local/include']]
return flags
__cxxFlags = [
'-Wundefined-func-template',
'-Wno-unused-local-typedef', # Bugs #2657 and #3209
]
def getDebugFlags(self, conf):
flags = super(ClangFlags, self).getDebugFlags(conf)
flags['CXXFLAGS'] += ['-fcolor-diagnostics',
'-Wundefined-func-template',
'-Wno-unused-local-typedef', # Bugs #2657 and #3209
]
if self.getCompilerVersion(conf) < (6, 0, 0):
flags['CXXFLAGS'] += ['-Wno-missing-braces'] # Bug #4721
flags = super().getDebugFlags(conf)
flags['CXXFLAGS'] += self.__cxxFlags
return flags
def getOptimizedFlags(self, conf):
flags = super(ClangFlags, self).getOptimizedFlags(conf)
flags['CXXFLAGS'] += ['-fcolor-diagnostics',
'-Wundefined-func-template',
'-Wno-unused-local-typedef', # Bugs #2657 and #3209
]
if self.getCompilerVersion(conf) < (6, 0, 0):
flags['CXXFLAGS'] += ['-Wno-missing-braces'] # Bug #4721
flags = super().getOptimizedFlags(conf)
flags['CXXFLAGS'] += self.__cxxFlags
return flags
+5 -3
View File
@@ -1,10 +1,12 @@
# PSync Authors
The following lists maintainers, primary developers, and all much-appreciated contributors to PSync in alphabetic order.
The following lists maintainers, primary developers, and all much-appreciated contributors to PSync in alphabetical order.
The specific contributions of individual authors can be obtained from the git history of the [official PSync repository](https://github.com/named-data/PSync).
If you would like to become a contributor to the official repository, please follow the recommendations in https://github.com/named-data/.github/blob/master/CONTRIBUTING.md.
If you would like to become a contributor to the official repository, please follow the recommendations in <https://github.com/named-data/.github/blob/main/CONTRIBUTING.md>.
* ***(Maintainer)*** Ashlesh Gawande <https://www.linkedin.com/in/agawande>
* Alexander Afanasyev <https://users.cs.fiu.edu/~afanasyev>
* ***(Maintainer)*** Saurab Dulal <https://dulalsaurab.github.io>
* ***(Former Maintainer)*** Ashlesh Gawande <https://www.linkedin.com/in/agawande>
* Eric Newberry <https://ericnewberry.com>
* Davide Pesavento <https://github.com/Pesa>
* Junxiao Shi <https://cs.arizona.edu/~shijunxiao>
+1 -2
View File
@@ -18,9 +18,8 @@ PSync relies on third-party software, licensed under the following licenses:
- The waf build system is licensed under the terms of the
[BSD license](https://github.com/named-data/PSync/blob/master/waf)
The LGPL and GPL licenses are provided below in this file. For more information
about these licenses, see https://www.gnu.org/licenses/
about these licenses, see <https://www.gnu.org/licenses/>
--------------------------------------------------------------------------------
+6 -5
View File
@@ -1,6 +1,6 @@
/* -*- Mode:C++; c-file-style:"gnu"; indent-tabs-mode:nil; -*- */
/*
* Copyright (c) 2014-2021, The University of Memphis
* Copyright (c) 2014-2022, The University of Memphis
*
* This file is part of PSync.
* See AUTHORS.md for complete list of PSync authors and contributors.
@@ -35,10 +35,10 @@ namespace psync {
using namespace ndn::time_literals;
const ndn::time::milliseconds HELLO_INTEREST_LIFETIME = 1_s;
const ndn::time::milliseconds HELLO_REPLY_FRESHNESS = 1_s;
const ndn::time::milliseconds SYNC_INTEREST_LIFETIME = 1_s;
const ndn::time::milliseconds SYNC_REPLY_FRESHNESS = 1_s;
inline constexpr ndn::time::milliseconds HELLO_INTEREST_LIFETIME = 1_s;
inline constexpr ndn::time::milliseconds HELLO_REPLY_FRESHNESS = 1_s;
inline constexpr ndn::time::milliseconds SYNC_INTEREST_LIFETIME = 1_s;
inline constexpr ndn::time::milliseconds SYNC_REPLY_FRESHNESS = 1_s;
enum class CompressionScheme {
NONE,
@@ -65,6 +65,7 @@ struct MissingDataInfo
ndn::Name prefix;
uint64_t lowSeq;
uint64_t highSeq;
uint64_t incomingFace;
};
using UpdateCallback = std::function<void(const std::vector<MissingDataInfo>&)>;
+52 -23
View File
@@ -1,6 +1,6 @@
/* -*- Mode:C++; c-file-style:"gnu"; indent-tabs-mode:nil; -*- */
/*
* Copyright (c) 2014-2020, The University of Memphis
* Copyright (c) 2014-2023, The University of Memphis
*
* This file is part of PSync.
* See AUTHORS.md for complete list of PSync authors and contributors.
@@ -27,27 +27,33 @@ namespace psync {
NDN_LOG_INIT(psync.Consumer);
Consumer::Consumer(ndn::Face& face, const ndn::Name& syncPrefix, const Options& opts)
: m_face(face)
, m_scheduler(m_face.getIoContext())
, m_syncPrefix(syncPrefix)
, m_helloInterestPrefix(ndn::Name(m_syncPrefix).append("hello"))
, m_syncInterestPrefix(ndn::Name(m_syncPrefix).append("sync"))
, m_syncDataContentType(ndn::tlv::ContentType_Blob)
, m_onReceiveHelloData(opts.onHelloData)
, m_onUpdate(opts.onUpdate)
, m_bloomFilter(opts.bfCount, opts.bfFalsePositive)
, m_helloInterestLifetime(opts.helloInterestLifetime)
, m_syncInterestLifetime(opts.syncInterestLifetime)
, m_rng(ndn::random::getRandomNumberEngine())
, m_rangeUniformRandom(100, 500)
{
}
Consumer::Consumer(const ndn::Name& syncPrefix,
ndn::Face& face,
const ReceiveHelloCallback& onReceiveHelloData,
const UpdateCallback& onUpdate,
unsigned int count,
double false_positive = 0.001,
double falsePositive,
ndn::time::milliseconds helloInterestLifetime,
ndn::time::milliseconds syncInterestLifetime)
: m_face(face)
, m_scheduler(m_face.getIoService())
, m_syncPrefix(syncPrefix)
, m_helloInterestPrefix(ndn::Name(m_syncPrefix).append("hello"))
, m_syncInterestPrefix(ndn::Name(m_syncPrefix).append("sync"))
, m_syncDataContentType(ndn::tlv::ContentType_Blob)
, m_onReceiveHelloData(onReceiveHelloData)
, m_onUpdate(onUpdate)
, m_bloomFilter(count, false_positive)
, m_helloInterestLifetime(helloInterestLifetime)
, m_syncInterestLifetime(syncInterestLifetime)
, m_rng(ndn::random::getRandomNumberEngine())
, m_rangeUniformRandom(100, 500)
: Consumer(face, syncPrefix,
Options{onReceiveHelloData, onUpdate, count, falsePositive, helloInterestLifetime, syncInterestLifetime})
{
}
@@ -59,19 +65,42 @@ Consumer::addSubscription(const ndn::Name& prefix, uint64_t seqNo, bool callSync
return false;
}
NDN_LOG_DEBUG("Subscribing prefix: " << prefix);
m_subscriptionList.emplace(prefix);
m_bloomFilter.insert(prefix.toUri());
m_bloomFilter.insert(prefix);
if (callSyncDataCb && seqNo != 0) {
m_onUpdate({{prefix, seqNo, seqNo}});
m_onUpdate({{prefix, seqNo, seqNo, 0}});
}
return true;
}
bool
Consumer::removeSubscription(const ndn::Name& prefix)
{
if (!isSubscribed(prefix))
return false;
NDN_LOG_DEBUG("Unsubscribing prefix: " << prefix);
m_prefixes.erase(prefix);
m_subscriptionList.erase(prefix);
// Clear and reconstruct the bloom filter
m_bloomFilter.clear();
for (const auto& item : m_subscriptionList)
m_bloomFilter.insert(item);
return true;
}
void
Consumer::stop()
{
NDN_LOG_DEBUG("Canceling all the scheduled events");
m_scheduler.cancelAllEvents();
if (m_syncFetcher) {
@@ -95,7 +124,7 @@ Consumer::sendHelloInterest()
m_helloFetcher->stop();
}
using ndn::util::SegmentFetcher;
using ndn::SegmentFetcher;
SegmentFetcher::Options options;
options.interestLifetime = m_helloInterestLifetime;
options.maxTimeout = m_helloInterestLifetime;
@@ -130,7 +159,7 @@ Consumer::onHelloData(const ndn::ConstBufferPtr& bufferPtr)
// Extract IBF from name which is the last element in hello data's name
m_iblt = m_helloDataName.getSubName(m_helloDataName.size() - 1, 1);
NDN_LOG_TRACE("m_iblt: " << std::hash<std::string>{}(m_iblt.toUri()));
NDN_LOG_TRACE("m_iblt: " << std::hash<ndn::Name>{}(m_iblt));
detail::State state{ndn::Block(bufferPtr)};
std::vector<MissingDataInfo> updates;
@@ -145,7 +174,7 @@ Consumer::onHelloData(const ndn::ConstBufferPtr& bufferPtr)
// m_prefixes (see addSubscription). So [] operator is safe to use.
if (isSubscribed(prefix) && seq > m_prefixes[prefix]) {
// In case we are behind on this prefix and consumer is subscribed to it
updates.push_back({prefix, m_prefixes[prefix] + 1, seq});
updates.push_back({prefix, m_prefixes[prefix] + 1, seq, 0});
m_prefixes[prefix] = seq;
}
availableSubscriptions.emplace(prefix, seq);
@@ -175,13 +204,13 @@ Consumer::sendSyncInterest()
ndn::Interest syncInterest(syncInterestName);
NDN_LOG_DEBUG("sendSyncInterest, nonce: " << syncInterest.getNonce() <<
" hash: " << std::hash<std::string>{}(syncInterest.getName().toUri()));
" hash: " << std::hash<ndn::Name>{}(syncInterest.getName()));
if (m_syncFetcher) {
m_syncFetcher->stop();
}
using ndn::util::SegmentFetcher;
using ndn::SegmentFetcher;
SegmentFetcher::Options options;
options.interestLifetime = m_syncInterestLifetime;
options.maxTimeout = m_syncInterestLifetime;
@@ -240,7 +269,7 @@ Consumer::onSyncData(const ndn::ConstBufferPtr& bufferPtr)
if (m_prefixes.find(prefix) == m_prefixes.end() || seq > m_prefixes[prefix]) {
// If this is just the next seq number then we had already informed the consumer about
// the previous sequence number and hence seq low and seq high should be equal to current seq
updates.push_back({prefix, m_prefixes[prefix] + 1, seq});
updates.push_back({prefix, m_prefixes[prefix] + 1, seq, 0});
m_prefixes[prefix] = seq;
}
// Else updates will be empty and consumer will not be notified.
+42 -16
View File
@@ -1,6 +1,6 @@
/* -*- Mode:C++; c-file-style:"gnu"; indent-tabs-mode:nil; -*- */
/*
* Copyright (c) 2014-2021, The University of Memphis
* Copyright (c) 2014-2023, The University of Memphis
*
* This file is part of PSync.
* See AUTHORS.md for complete list of PSync authors and contributors.
@@ -56,23 +56,40 @@ class Consumer
{
public:
/**
* @brief constructor
*
* @param syncPrefix syncPrefix to send hello/sync interests to producer
* @param face application's face
* @param onReceiveHelloData call back to give hello data back to application
* @param onUpdate call back to give sync data back to application
* @param count bloom filter number of expected elements (subscriptions) in bloom filter
* @param false_positive bloom filter false positive probability
* @param helloInterestLifetime lifetime of hello interest
* @param syncInterestLifetime lifetime of sync interest
* @brief Constructor options.
*/
struct Options
{
/// Callback to give hello data back to application.
ReceiveHelloCallback onHelloData = [] (const auto&) {};
/// Callback to give sync data back to application.
UpdateCallback onUpdate = [] (const auto&) {};
/// Number of expected elements (subscriptions) in Bloom filter.
unsigned int bfCount = 80;
/// Bloom filter false positive probability.
double bfFalsePositive = 0.001;
/// Lifetime of hello Interest.
ndn::time::milliseconds helloInterestLifetime = HELLO_INTEREST_LIFETIME;
/// Lifetime of sync Interest.
ndn::time::milliseconds syncInterestLifetime = SYNC_INTEREST_LIFETIME;
};
/**
* @brief Constructor.
*
* @param face Application face.
* @param syncPrefix Prefix to send hello and sync Interests to producer.
* @param opts Options.
*/
Consumer(ndn::Face& face, const ndn::Name& syncPrefix, const Options& opts);
[[deprecated]]
Consumer(const ndn::Name& syncPrefix,
ndn::Face& face,
const ReceiveHelloCallback& onReceiveHelloData,
const UpdateCallback& onUpdate,
unsigned int count,
double false_positive,
double falsePositive = 0.001,
ndn::time::milliseconds helloInterestLifetime = HELLO_INTEREST_LIFETIME,
ndn::time::milliseconds syncInterestLifetime = SYNC_INTEREST_LIFETIME);
@@ -107,6 +124,15 @@ public:
bool
addSubscription(const ndn::Name& prefix, uint64_t seqNo, bool callSyncDataCb = true);
/**
* @brief Remove prefix from subscription list
*
* @param prefix prefix to be removed from the list
* @return true if prefix is removed, false if it is not present
*/
bool
removeSubscription(const ndn::Name& prefix);
std::set<ndn::Name>
getSubscriptionList() const
{
@@ -119,12 +145,12 @@ public:
return m_subscriptionList.find(prefix) != m_subscriptionList.end();
}
ndn::optional<uint64_t>
std::optional<uint64_t>
getSeqNo(const ndn::Name& prefix) const
{
auto it = m_prefixes.find(prefix);
if (it == m_prefixes.end()) {
return ndn::nullopt;
return std::nullopt;
}
return it->second;
}
@@ -193,8 +219,8 @@ PSYNC_PUBLIC_WITH_TESTS_ELSE_PRIVATE:
ndn::random::RandomNumberEngine& m_rng;
std::uniform_int_distribution<> m_rangeUniformRandom;
std::shared_ptr<ndn::util::SegmentFetcher> m_helloFetcher;
std::shared_ptr<ndn::util::SegmentFetcher> m_syncFetcher;
std::shared_ptr<ndn::SegmentFetcher> m_helloFetcher;
std::shared_ptr<ndn::SegmentFetcher> m_syncFetcher;
};
} // namespace psync
+5 -7
View File
@@ -1,6 +1,6 @@
/* -*- Mode:C++; c-file-style:"gnu"; indent-tabs-mode:nil; -*- */
/*
* Copyright (c) 2014-2020, The University of Memphis
* Copyright (c) 2014-2022, The University of Memphis
*
* This file is part of PSync.
* See AUTHORS.md for complete list of PSync authors and contributors.
@@ -53,8 +53,7 @@
#include <cstdlib>
#include <limits>
namespace psync {
namespace detail {
namespace psync::detail {
// https://github.com/ArashPartow/bloom
@@ -250,7 +249,7 @@ BloomFilter::clear()
}
void
BloomFilter::insert(const std::string& key)
BloomFilter::insert(const ndn::Name& key)
{
std::size_t bit_index = 0;
std::size_t bit = 0;
@@ -266,7 +265,7 @@ BloomFilter::insert(const std::string& key)
}
bool
BloomFilter::contains(const std::string& key) const
BloomFilter::contains(const ndn::Name& key) const
{
std::size_t bit_index = 0;
std::size_t bit = 0;
@@ -376,5 +375,4 @@ BloomFilter::generate_unique_salt()
}
}
} // namespace detail
} // namespace psync
} // namespace psync::detail
+6 -8
View File
@@ -1,6 +1,6 @@
/* -*- Mode:C++; c-file-style:"gnu"; indent-tabs-mode:nil; -*- */
/*
* Copyright (c) 2014-2020, The University of Memphis
* Copyright (c) 2014-2022, The University of Memphis
*
* This file is part of PSync.
* See AUTHORS.md for complete list of PSync authors and contributors.
@@ -52,8 +52,7 @@
#include <string>
#include <vector>
namespace psync {
namespace detail {
namespace psync::detail {
class bloom_parameters;
@@ -90,10 +89,10 @@ public:
clear();
void
insert(const std::string& key);
insert(const ndn::Name& key);
bool
contains(const std::string& key) const;
contains(const ndn::Name& key) const;
private:
typedef uint32_t bloom_type;
@@ -127,7 +126,7 @@ private: // non-member operators
friend std::ostream&
operator<<(std::ostream& os, const BloomFilter& bf)
{
ndn::printHex(os, bf.bit_table_.data(), bf.bit_table_.size(), false);
ndn::printHex(os, bf.bit_table_, false);
return os;
}
@@ -142,7 +141,6 @@ private:
double desired_false_positive_probability_ = 0.0;
};
} // namespace detail
} // namespace psync
} // namespace psync::detail
#endif // PSYNC_DETAIL_BLOOM_FILTER_HPP
+6 -11
View File
@@ -1,6 +1,6 @@
/* -*- Mode:C++; c-file-style:"gnu"; indent-tabs-mode:nil; -*- */
/*
* Copyright (c) 2014-2020, The University of Memphis
* Copyright (c) 2014-2022, The University of Memphis
*
* This file is part of PSync.
* See AUTHORS.md for complete list of PSync authors and contributors.
@@ -48,14 +48,10 @@
#include <ndn-cxx/util/exception.hpp>
namespace psync {
namespace detail {
namespace psync::detail {
namespace be = boost::endian;
const size_t N_HASH(3);
const size_t N_HASHCHECK(11);
bool
HashTableEntry::isPure() const
{
@@ -158,7 +154,7 @@ IBLT::listEntries(std::set<uint32_t>& positive, std::set<uint32_t>& negative) co
// If any buckets for one of the hash functions is not empty,
// then we didn't peel them all:
for (const auto& entry : peeled.m_hashTable) {
if (entry.isEmpty() != true) {
if (!entry.isEmpty()) {
return false;
}
}
@@ -203,14 +199,14 @@ IBLT::appendToName(ndn::Name& name) const
std::memcpy(&table[(i * unitSize) + 8], &keyCheck, sizeof(keyCheck));
}
auto compressed = compress(m_compressionScheme, table.data(), table.size());
auto compressed = compress(m_compressionScheme, table);
name.append(ndn::name::Component(std::move(compressed)));
}
std::vector<uint32_t>
IBLT::extractValueFromName(const ndn::name::Component& ibltName) const
{
auto decompressedBuf = decompress(m_compressionScheme, ibltName.value(), ibltName.value_size());
auto decompressedBuf = decompress(m_compressionScheme, ibltName.value_bytes());
if (decompressedBuf->size() % 4 != 0) {
NDN_THROW(Error("Received IBF cannot be decompressed correctly!"));
@@ -267,5 +263,4 @@ operator<<(std::ostream& os, const IBLT& iblt)
return os;
}
} // namespace detail
} // namespace psync
} // namespace psync::detail
+7 -9
View File
@@ -1,6 +1,6 @@
/* -*- Mode:C++; c-file-style:"gnu"; indent-tabs-mode:nil; -*- */
/*
* Copyright (c) 2014-2020, The University of Memphis
* Copyright (c) 2014-2022, The University of Memphis
*
* This file is part of PSync.
* See AUTHORS.md for complete list of PSync authors and contributors.
@@ -53,8 +53,7 @@
#include <set>
#include <string>
namespace psync {
namespace detail {
namespace psync::detail {
class HashTableEntry
{
@@ -70,8 +69,8 @@ public:
isEmpty() const;
};
extern const size_t N_HASH;
extern const size_t N_HASHCHECK;
inline constexpr size_t N_HASH = 3;
inline constexpr size_t N_HASHCHECK = 11;
/**
* @brief Invertible Bloom Lookup Table (Invertible Bloom Filter)
@@ -162,8 +161,8 @@ private:
private:
std::vector<HashTableEntry> m_hashTable;
static const int INSERT = 1;
static const int ERASE = -1;
static constexpr int INSERT = 1;
static constexpr int ERASE = -1;
CompressionScheme m_compressionScheme;
};
@@ -176,7 +175,6 @@ operator!=(const IBLT& iblt1, const IBLT& iblt2);
std::ostream&
operator<<(std::ostream& os, const IBLT& iblt);
} // namespace detail
} // namespace psync
} // namespace psync::detail
#endif // PSYNC_DETAIL_IBLT_HPP
+5 -9
View File
@@ -1,6 +1,6 @@
/* -*- Mode:C++; c-file-style:"gnu"; indent-tabs-mode:nil; -*- */
/*
* Copyright (c) 2014-2020, The University of Memphis
* Copyright (c) 2014-2022, The University of Memphis
*
* This file is part of PSync.
* See AUTHORS.md for complete list of PSync authors and contributors.
@@ -22,8 +22,7 @@
#include <ndn-cxx/util/exception.hpp>
#include <ndn-cxx/util/ostream-joiner.hpp>
namespace psync {
namespace detail {
namespace psync::detail {
State::State(const ndn::Block& block)
{
@@ -76,8 +75,7 @@ void
State::wireDecode(const ndn::Block& wire)
{
if (wire.type() != tlv::PSyncContent) {
NDN_THROW(ndn::tlv::Error("Expected PSyncContent element, but TLV has type " +
ndn::to_string(wire.type())));
NDN_THROW(ndn::tlv::Error("PSyncContent", wire.type()));
}
m_content.clear();
@@ -90,8 +88,7 @@ State::wireDecode(const ndn::Block& wire)
m_content.emplace_back(*it);
}
else {
NDN_THROW(ndn::tlv::Error("Expected Name element, but TLV has type " +
ndn::to_string(it->type())));
NDN_THROW(ndn::tlv::Error("Name", it->type()));
}
}
}
@@ -108,5 +105,4 @@ operator<<(std::ostream& os, const State& state)
return os;
}
} // namespace detail
} // namespace psync
} // namespace psync::detail
+5 -7
View File
@@ -1,6 +1,6 @@
/* -*- Mode:C++; c-file-style:"gnu"; indent-tabs-mode:nil; -*- */
/*
* Copyright (c) 2014-2020, The University of Memphis
* Copyright (c) 2014-2022, The University of Memphis
*
* This file is part of PSync.
* See AUTHORS.md for complete list of PSync authors and contributors.
@@ -22,16 +22,15 @@
#include <ndn-cxx/name.hpp>
namespace psync {
namespace tlv {
namespace psync::tlv {
enum {
PSyncContent = 128
};
} // namespace tlv
} // namespace psync::tlv
namespace detail {
namespace psync::detail {
class State
{
@@ -82,7 +81,6 @@ NDN_CXX_DECLARE_WIRE_ENCODE_INSTANTIATIONS(State);
std::ostream&
operator<<(std::ostream& os, const State& State);
} // namespace detail
} // namespace psync
} // namespace psync::detail
#endif // PSYNC_DETAIL_STATE_HPP
+33 -24
View File
@@ -1,6 +1,6 @@
/* -*- Mode:C++; c-file-style:"gnu"; indent-tabs-mode:nil; -*- */
/*
* Copyright (c) 2014-2020, The University of Memphis
* Copyright (c) 2014-2022, The University of Memphis
*
* This file is part of PSync.
* See AUTHORS.md for complete list of PSync authors and contributors.
@@ -46,10 +46,7 @@
#include <boost/iostreams/filter/zstd.hpp>
#endif
namespace psync {
namespace detail {
namespace bio = boost::iostreams;
namespace psync::detail {
static inline uint32_t
ROTL32(uint32_t x, int8_t r)
@@ -96,9 +93,9 @@ murmurHash3(const void* key, size_t len, uint32_t seed)
switch (len & 3)
{
case 3: k1 ^= tail[2] << 16;
NDN_CXX_FALLTHROUGH;
[[fallthrough]];
case 2: k1 ^= tail[1] << 8;
NDN_CXX_FALLTHROUGH;
[[fallthrough]];
case 1: k1 ^= tail[0];
k1 *= c1; k1 = ROTL32(k1,15); k1 *= c2; h1 ^= k1;
}
@@ -116,13 +113,24 @@ murmurHash3(const void* key, size_t len, uint32_t seed)
return h1;
}
std::shared_ptr<ndn::Buffer>
compress(CompressionScheme scheme, const uint8_t* buffer, size_t bufferSize)
uint32_t
murmurHash3(uint32_t seed, const ndn::Name& name)
{
ndn::OBufferStream out;
bio::filtering_streambuf<bio::input> in;
const auto& wire = name.wireEncode();
return murmurHash3(wire.value(), wire.value_size(), seed);
}
std::shared_ptr<ndn::Buffer>
compress(CompressionScheme scheme, ndn::span<const uint8_t> buffer)
{
namespace bio = boost::iostreams;
bio::filtering_istreambuf in;
switch (scheme) {
case CompressionScheme::NONE:
break;
case CompressionScheme::ZLIB:
#ifdef PSYNC_HAVE_ZLIB
in.push(bio::zlib_compressor(bio::zlib::best_compression));
@@ -162,23 +170,26 @@ compress(CompressionScheme scheme, const uint8_t* buffer, size_t bufferSize)
#else
NDN_THROW(CompressionError("ZSTD compression not supported!"));
#endif
case CompressionScheme::NONE:
break;
}
in.push(bio::array_source(reinterpret_cast<const char*>(buffer), bufferSize));
in.push(bio::array_source(reinterpret_cast<const char*>(buffer.data()), buffer.size()));
ndn::OBufferStream out;
bio::copy(in, out);
return out.buf();
}
std::shared_ptr<ndn::Buffer>
decompress(CompressionScheme scheme, const uint8_t* buffer, size_t bufferSize)
decompress(CompressionScheme scheme, ndn::span<const uint8_t> buffer)
{
ndn::OBufferStream out;
bio::filtering_streambuf<bio::input> in;
namespace bio = boost::iostreams;
bio::filtering_istreambuf in;
switch (scheme) {
case CompressionScheme::NONE:
break;
case CompressionScheme::ZLIB:
#ifdef PSYNC_HAVE_ZLIB
in.push(bio::zlib_decompressor());
@@ -218,15 +229,13 @@ decompress(CompressionScheme scheme, const uint8_t* buffer, size_t bufferSize)
#else
NDN_THROW(CompressionError("ZSTD compression not supported!"));
#endif
case CompressionScheme::NONE:
break;
}
in.push(bio::array_source(reinterpret_cast<const char*>(buffer), bufferSize));
in.push(bio::array_source(reinterpret_cast<const char*>(buffer.data()), buffer.size()));
ndn::OBufferStream out;
bio::copy(in, out);
return out.buf();
}
} // namespace detail
} // namespace psync
} // namespace psync::detail
+11 -14
View File
@@ -1,6 +1,6 @@
/* -*- Mode:C++; c-file-style:"gnu"; indent-tabs-mode:nil; -*- */
/*
* Copyright (c) 2014-2020, The University of Memphis
* Copyright (c) 2014-2022, The University of Memphis
*
* This file is part of PSync.
* See AUTHORS.md for complete list of PSync authors and contributors.
@@ -23,20 +23,18 @@
#include "PSync/common.hpp"
#include <ndn-cxx/encoding/buffer.hpp>
#include <ndn-cxx/util/span.hpp>
#include <string>
namespace psync {
namespace detail {
namespace psync::detail {
uint32_t
murmurHash3(const void* key, size_t len, uint32_t seed);
inline uint32_t
murmurHash3(uint32_t seed, const std::string& str)
{
return murmurHash3(str.data(), str.size(), seed);
}
/**
* @brief Compute 32-bit MurmurHash3 of Name TLV-VALUE.
*/
uint32_t
murmurHash3(uint32_t seed, const ndn::Name& name);
inline uint32_t
murmurHash3(uint32_t seed, uint32_t value)
@@ -45,12 +43,11 @@ murmurHash3(uint32_t seed, uint32_t value)
}
std::shared_ptr<ndn::Buffer>
compress(CompressionScheme scheme, const uint8_t* buffer, size_t bufferSize);
compress(CompressionScheme scheme, ndn::span<const uint8_t> buffer);
std::shared_ptr<ndn::Buffer>
decompress(CompressionScheme scheme, const uint8_t* buffer, size_t bufferSize);
decompress(CompressionScheme scheme, ndn::span<const uint8_t> buffer);
} // namespace detail
} // namespace psync
} // namespace psync::detail
#endif // PSYNC_DETAIL_UTIL_HPP
+40 -28
View File
@@ -1,6 +1,6 @@
/* -*- Mode:C++; c-file-style:"gnu"; indent-tabs-mode:nil; -*- */
/*
* Copyright (c) 2014-2020, The University of Memphis
* Copyright (c) 2014-2023, The University of Memphis
*
* This file is part of PSync.
* See AUTHORS.md for complete list of PSync authors and contributors.
@@ -21,9 +21,9 @@
#include "PSync/detail/state.hpp"
#include "PSync/detail/util.hpp"
#include <ndn-cxx/lp/tags.hpp>
#include <ndn-cxx/security/validator-null.hpp>
#include <ndn-cxx/util/logger.hpp>
#include <ndn-cxx/util/segment-fetcher.hpp>
#include <cstring>
@@ -31,25 +31,24 @@ namespace psync {
NDN_LOG_INIT(psync.FullProducer);
FullProducer::FullProducer(const size_t expectedNumEntries,
ndn::Face& face,
FullProducer::FullProducer(ndn::Face& face,
ndn::KeyChain& keyChain,
size_t expectedNumEntries,
const ndn::Name& syncPrefix,
const ndn::Name& userPrefix,
const UpdateCallback& onUpdateCallBack,
UpdateCallback onUpdateCb,
ndn::time::milliseconds syncInterestLifetime,
ndn::time::milliseconds syncReplyFreshness,
CompressionScheme ibltCompression,
CompressionScheme contentCompression)
: ProducerBase(expectedNumEntries, face, syncPrefix, userPrefix, syncReplyFreshness,
ibltCompression, contentCompression)
: ProducerBase(face, keyChain, expectedNumEntries, syncPrefix, userPrefix,
syncReplyFreshness, ibltCompression, contentCompression)
, m_syncInterestLifetime(syncInterestLifetime)
, m_onUpdate(onUpdateCallBack)
, m_jitter(100, 500)
, m_onUpdate(std::move(onUpdateCb))
{
m_registeredPrefix = m_face.setInterestFilter(
ndn::InterestFilter(m_syncPrefix).allowLoopback(false),
std::bind(&FullProducer::onSyncInterest, this, _1, _2),
std::bind(&FullProducer::onRegisterFailed, this, _1, _2));
m_registeredPrefix = m_face.setInterestFilter(ndn::InterestFilter(m_syncPrefix).allowLoopback(false),
[this] (auto&&... args) { onSyncInterest(std::forward<decltype(args)>(args)...); },
[] (auto&&... args) { onRegisterFailed(std::forward<decltype(args)>(args)...); });
// Should we do this after setInterestFilter success call back
// (Currently following ChronoSync's way)
@@ -64,7 +63,7 @@ FullProducer::~FullProducer()
}
void
FullProducer::publishName(const ndn::Name& prefix, ndn::optional<uint64_t> seq)
FullProducer::publishName(const ndn::Name& prefix, std::optional<uint64_t> seq)
{
if (m_prefixes.find(prefix) == m_prefixes.end()) {
NDN_LOG_WARN("Prefix not added: " << prefix);
@@ -72,11 +71,8 @@ FullProducer::publishName(const ndn::Name& prefix, ndn::optional<uint64_t> seq)
}
uint64_t newSeq = seq.value_or(m_prefixes[prefix] + 1);
NDN_LOG_INFO("Publish: " << prefix << "/" << newSeq);
updateSeqNo(prefix, newSeq);
satisfyPendingInterests();
}
@@ -104,7 +100,7 @@ FullProducer::sendSyncInterest()
ndn::Interest syncInterest(syncInterestName);
using ndn::util::SegmentFetcher;
using ndn::SegmentFetcher;
SegmentFetcher::Options options;
options.interestLifetime = m_syncInterestLifetime;
options.maxTimeout = m_syncInterestLifetime;
@@ -117,6 +113,16 @@ FullProducer::sendSyncInterest()
onSyncData(syncInterest, bufferPtr);
});
m_fetcher->afterSegmentValidated.connect([this] (const ndn::Data& data) {
auto tag = data.getTag<ndn::lp::IncomingFaceIdTag>();
if (tag) {
m_incomingFace = *tag;
}
else {
m_incomingFace = 0;
}
});
m_fetcher->onError.connect([this] (uint32_t errorCode, const std::string& msg) {
NDN_LOG_ERROR("Cannot fetch sync data, error: " << errorCode << ", message: " << msg);
// We would like to recover from errors like NoRoute NACK quicker than sync Interest timeout.
@@ -184,7 +190,7 @@ FullProducer::onSyncInterest(const ndn::Name& prefixName, const ndn::Interest& i
// Send all data if greater then threshold, else send positive below as usual
// Or send if we can't get neither positive nor negative differences
if (positive.size() + negative.size() >= m_threshold ||
(positive.size() == 0 && negative.size() == 0)) {
(positive.empty() && negative.empty())) {
detail::State state;
for (const auto& content : m_prefixes) {
if (content.second != 0) {
@@ -196,8 +202,15 @@ FullProducer::onSyncInterest(const ndn::Name& prefixName, const ndn::Interest& i
sendSyncData(interest.getName(), state.wireEncode());
}
#ifdef PSYNC_WITH_TESTS
++nIbfDecodeFailuresAboveThreshold;
#endif // PSYNC_WITH_TESTS
return;
}
#ifdef PSYNC_WITH_TESTS
++nIbfDecodeFailuresBelowThreshold;
#endif // PSYNC_WITH_TESTS
}
detail::State state;
@@ -207,7 +220,7 @@ FullProducer::onSyncInterest(const ndn::Name& prefixName, const ndn::Interest& i
ndn::Name nameWithoutSeq = nameIt->second.getPrefix(-1);
// Don't sync up sequence number zero
if (m_prefixes[nameWithoutSeq] != 0 &&
!isFutureHash(nameWithoutSeq.toUri(), negative)) {
!isFutureHash(nameWithoutSeq, negative)) {
state.addContent(nameIt->second);
}
}
@@ -238,7 +251,7 @@ FullProducer::sendSyncData(const ndn::Name& name, const ndn::Block& block)
// TODO: Remove appending of hash - serves no purpose to the receiver
ndn::Name dataName(ndn::Name(name).appendNumber(std::hash<ndn::Name>{}(nameWithIblt)));
auto content = detail::compress(m_contentCompression, block.wire(), block.size());
auto content = detail::compress(m_contentCompression, block);
// checking if our own interest got satisfied
if (m_outstandingInterestName == name) {
@@ -253,14 +266,14 @@ FullProducer::sendSyncData(const ndn::Name& name, const ndn::Block& block)
NDN_LOG_DEBUG("Sending sync Data");
// Send data after removing pending sync interest on face
m_segmentPublisher.publish(name, dataName, content, m_syncReplyFreshness);
m_segmentPublisher.publish(name, dataName, *content, m_syncReplyFreshness);
NDN_LOG_TRACE("Renewing sync interest");
sendSyncInterest();
}
else {
NDN_LOG_DEBUG("Sending sync Data");
m_segmentPublisher.publish(name, dataName, content, m_syncReplyFreshness);
m_segmentPublisher.publish(name, dataName, *content, m_syncReplyFreshness);
}
}
@@ -271,14 +284,13 @@ FullProducer::onSyncData(const ndn::Interest& interest, const ndn::ConstBufferPt
detail::State state;
try {
auto decompressed = detail::decompress(m_contentCompression, bufferPtr->data(), bufferPtr->size());
auto decompressed = detail::decompress(m_contentCompression, *bufferPtr);
state.wireDecode(ndn::Block(std::move(decompressed)));
}
catch (const std::exception& e) {
NDN_LOG_ERROR("Cannot parse received sync Data: " << e.what());
return;
}
NDN_LOG_DEBUG("Sync Data received: " << state);
std::vector<MissingDataInfo> updates;
@@ -288,7 +300,7 @@ FullProducer::onSyncData(const ndn::Interest& interest, const ndn::ConstBufferPt
uint64_t seq = content.get(content.size() - 1).toNumber();
if (m_prefixes.find(prefix) == m_prefixes.end() || m_prefixes[prefix] < seq) {
updates.push_back({prefix, m_prefixes[prefix] + 1, seq});
updates.push_back({prefix, m_prefixes[prefix] + 1, seq, m_incomingFace});
updateSeqNo(prefix, seq);
// We should not call satisfyPendingSyncInterests here because we just
// got data and deleted pending interest by calling deletePendingFullSyncInterests
@@ -323,7 +335,7 @@ FullProducer::satisfyPendingInterests()
if (!diff.listEntries(positive, negative)) {
NDN_LOG_TRACE("Decode failed for pending interest");
if (positive.size() + negative.size() >= m_threshold ||
(positive.size() == 0 && negative.size() == 0)) {
(positive.empty() && negative.empty())) {
NDN_LOG_TRACE("pos + neg > threshold or no diff can be found, erase pending interest");
it = m_pendingEntries.erase(it);
continue;
@@ -355,7 +367,7 @@ bool
FullProducer::isFutureHash(const ndn::Name& prefix, const std::set<uint32_t>& negative)
{
auto nextHash = detail::murmurHash3(detail::N_HASHCHECK,
ndn::Name(prefix).appendNumber(m_prefixes[prefix] + 1).toUri());
ndn::Name(prefix).appendNumber(m_prefixes[prefix] + 1));
return negative.find(nextHash) != negative.end();
}
+23 -14
View File
@@ -1,6 +1,6 @@
/* -*- Mode:C++; c-file-style:"gnu"; indent-tabs-mode:nil; -*- */
/*
* Copyright (c) 2014-2021, The University of Memphis
* Copyright (c) 2014-2023, The University of Memphis
*
* This file is part of PSync.
* See AUTHORS.md for complete list of PSync authors and contributors.
@@ -42,25 +42,27 @@ class FullProducer : public ProducerBase
{
public:
/**
* @brief constructor
* @brief Constructor
*
* Registers syncPrefix in NFD and sends a sync interest
* Registers syncPrefix in NFD and sends a sync interest.
*
* @param expectedNumEntries expected entries in IBF
* @param face application's face
* @param face Application's face
* @param keyChain KeyChain instance to use for signing
* @param expectedNumEntries Expected number of entries in IBF
* @param syncPrefix The prefix of the sync group
* @param userPrefix The prefix of the first user in the group
* @param onUpdateCallBack The call back to be called when there is new data
* @param syncInterestLifetime lifetime of the sync interest
* @param syncReplyFreshness freshness of sync data
* @param onUpdateCallBack The callback to be invoked when there is new data
* @param syncInterestLifetime Lifetime of the sync interest
* @param syncReplyFreshness FreshnessPeriod of sync data
* @param ibltCompression Compression scheme to use for IBF
* @param contentCompression Compression scheme to use for Data content
*/
FullProducer(size_t expectedNumEntries,
ndn::Face& face,
FullProducer(ndn::Face& face,
ndn::KeyChain& keyChain,
size_t expectedNumEntries,
const ndn::Name& syncPrefix,
const ndn::Name& userPrefix,
const UpdateCallback& onUpdateCallBack,
UpdateCallback onUpdateCallBack,
ndn::time::milliseconds syncInterestLifetime = SYNC_INTEREST_LIFETIME,
ndn::time::milliseconds syncReplyFreshness = SYNC_REPLY_FRESHNESS,
CompressionScheme ibltCompression = CompressionScheme::DEFAULT,
@@ -80,7 +82,7 @@ public:
* @param seq the sequence number of the prefix
*/
void
publishName(const ndn::Name& prefix, ndn::optional<uint64_t> seq = ndn::nullopt);
publishName(const ndn::Name& prefix, std::optional<uint64_t> seq = std::nullopt);
PSYNC_PUBLIC_WITH_TESTS_ELSE_PRIVATE:
/**
@@ -167,6 +169,12 @@ private:
bool
isFutureHash(const ndn::Name& prefix, const std::set<uint32_t>& negative);
#ifdef PSYNC_WITH_TESTS
public:
size_t nIbfDecodeFailuresAboveThreshold = 0;
size_t nIbfDecodeFailuresBelowThreshold = 0;
#endif // PSYNC_WITH_TESTS
private:
struct PendingEntryInfo
{
@@ -178,10 +186,11 @@ private:
ndn::time::milliseconds m_syncInterestLifetime;
UpdateCallback m_onUpdate;
ndn::scheduler::ScopedEventId m_scheduledSyncInterestId;
std::uniform_int_distribution<> m_jitter;
std::uniform_int_distribution<> m_jitter{100, 500};
ndn::Name m_outstandingInterestName;
ndn::ScopedRegisteredPrefixHandle m_registeredPrefix;
std::shared_ptr<ndn::util::SegmentFetcher> m_fetcher;
std::shared_ptr<ndn::SegmentFetcher> m_fetcher;
uint64_t m_incomingFace = 0;
};
} // namespace psync
+22 -22
View File
@@ -1,6 +1,6 @@
/* -*- Mode:C++; c-file-style:"gnu"; indent-tabs-mode:nil; -*- */
/*
* Copyright (c) 2014-2020, The University of Memphis
* Copyright (c) 2014-2022, The University of Memphis
*
* This file is part of PSync.
* See AUTHORS.md for complete list of PSync authors and contributors.
@@ -23,60 +23,60 @@
#include <ndn-cxx/util/logger.hpp>
#include <cstring>
#include <limits>
namespace psync {
NDN_LOG_INIT(psync.PartialProducer);
PartialProducer::PartialProducer(size_t expectedNumEntries,
ndn::Face& face,
const ndn::name::Component HELLO{"hello"};
const ndn::name::Component SYNC{"sync"};
PartialProducer::PartialProducer(ndn::Face& face,
ndn::KeyChain& keyChain,
size_t expectedNumEntries,
const ndn::Name& syncPrefix,
const ndn::Name& userPrefix,
ndn::time::milliseconds helloReplyFreshness,
ndn::time::milliseconds syncReplyFreshness,
CompressionScheme ibltCompression)
: ProducerBase(expectedNumEntries, face, syncPrefix,
userPrefix, syncReplyFreshness, ibltCompression)
, m_helloReplyFreshness(helloReplyFreshness)
: ProducerBase(face, keyChain, expectedNumEntries, syncPrefix, userPrefix,
syncReplyFreshness, ibltCompression, CompressionScheme::NONE)
, m_helloReplyFreshness(helloReplyFreshness)
{
m_registeredPrefix = m_face.registerPrefix(m_syncPrefix,
[this] (const ndn::Name& syncPrefix) {
m_face.setInterestFilter(ndn::Name(m_syncPrefix).append("hello"),
[this] (const auto&) {
m_face.setInterestFilter(ndn::Name(m_syncPrefix).append(HELLO),
std::bind(&PartialProducer::onHelloInterest, this, _1, _2));
m_face.setInterestFilter(ndn::Name(m_syncPrefix).append("sync"),
m_face.setInterestFilter(ndn::Name(m_syncPrefix).append(SYNC),
std::bind(&PartialProducer::onSyncInterest, this, _1, _2));
},
std::bind(&PartialProducer::onRegisterFailed, this, _1, _2));
[] (auto&&... args) { onRegisterFailed(std::forward<decltype(args)>(args)...); });
}
void
PartialProducer::publishName(const ndn::Name& prefix, ndn::optional<uint64_t> seq)
PartialProducer::publishName(const ndn::Name& prefix, std::optional<uint64_t> seq)
{
if (m_prefixes.find(prefix) == m_prefixes.end()) {
return;
}
uint64_t newSeq = seq.value_or(m_prefixes[prefix] + 1);
NDN_LOG_INFO("Publish: " << prefix << "/" << newSeq);
updateSeqNo(prefix, newSeq);
satisfyPendingSyncInterests(prefix);
}
void
PartialProducer::onHelloInterest(const ndn::Name& prefix, const ndn::Interest& interest)
{
if (m_segmentPublisher.replyFromStore(interest.getName())) {
const auto& name = interest.getName();
if (m_segmentPublisher.replyFromStore(name)) {
return;
}
// Last component or fourth last component (in case of interest with version and segment)
// needs to be hello
if (interest.getName().get(interest.getName().size()-1).toUri() != "hello" &&
interest.getName().get(interest.getName().size()-4).toUri() != "hello") {
if (name.get(name.size() - 1) != HELLO && name.get(name.size() - 4) != HELLO) {
return;
}
@@ -103,7 +103,7 @@ PartialProducer::onSyncInterest(const ndn::Name& prefix, const ndn::Interest& in
}
NDN_LOG_DEBUG("Sync Interest Received, nonce: " << interest.getNonce() <<
" hash: " << std::hash<std::string>{}(interest.getName().toUri()));
" hash: " << std::hash<ndn::Name>{}(interest.getName()));
ndn::Name nameWithoutSyncPrefix = interest.getName().getSubName(prefix.size());
ndn::Name interestName;
@@ -173,7 +173,7 @@ PartialProducer::onSyncInterest(const ndn::Name& prefix, const ndn::Interest& in
for (const auto& hash : positive) {
auto nameIt = m_biMap.left.find(hash);
if (nameIt != m_biMap.left.end()) {
if (bf.contains(nameIt->second.getPrefix(-1).toUri())) {
if (bf.contains(nameIt->second.getPrefix(-1))) {
// generate data
state.addContent(nameIt->second);
NDN_LOG_DEBUG("Content: " << nameIt->second << " " << nameIt->first);
@@ -227,8 +227,8 @@ PartialProducer::satisfyPendingSyncInterests(const ndn::Name& prefix) {
}
detail::State state;
if (entry.bf.contains(prefix.toUri()) || positive.size() + negative.size() >= m_threshold) {
if (entry.bf.contains(prefix.toUri())) {
if (entry.bf.contains(prefix) || positive.size() + negative.size() >= m_threshold) {
if (entry.bf.contains(prefix)) {
state.addContent(ndn::Name(prefix).appendNumber(m_prefixes[prefix]));
NDN_LOG_DEBUG("sending sync content " << prefix << " " << std::to_string(m_prefixes[prefix]));
}
+13 -17
View File
@@ -1,6 +1,6 @@
/* -*- Mode:C++; c-file-style:"gnu"; indent-tabs-mode:nil; -*- */
/*
* Copyright (c) 2014-2021, The University of Memphis
* Copyright (c) 2014-2022, The University of Memphis
*
* This file is part of PSync.
* See AUTHORS.md for complete list of PSync authors and contributors.
@@ -20,14 +20,8 @@
#ifndef PSYNC_PARTIAL_PRODUCER_HPP
#define PSYNC_PARTIAL_PRODUCER_HPP
#include "PSync/detail/bloom-filter.hpp"
#include "PSync/producer-base.hpp"
#include <map>
#include <ndn-cxx/face.hpp>
#include <ndn-cxx/security/key-chain.hpp>
#include <ndn-cxx/util/scheduler.hpp>
#include "PSync/detail/bloom-filter.hpp"
namespace psync {
@@ -43,21 +37,23 @@ class PartialProducer : public ProducerBase
{
public:
/**
* @brief constructor
* @brief Constructor
*
* Registers syncPrefix in NFD and sets internal filters for
* "sync" and "hello" under syncPrefix
* "sync" and "hello" under syncPrefix.
*
* @param expectedNumEntries expected entries in IBF
* @param face application's face
* @param face Application's face
* @param keyChain KeyChain instance to use for signing
* @param expectedNumEntries Expected number of entries in IBF
* @param syncPrefix The prefix of the sync group
* @param userPrefix The prefix of the first user in the group
* @param syncReplyFreshness freshness of sync data
* @param helloReplyFreshness freshness of hello data
* @param helloReplyFreshness FreshnessPeriod of hello data
* @param syncReplyFreshness FreshnessPeriod of sync data
* @param ibltCompression Compression scheme to use for IBF
*/
PartialProducer(size_t expectedNumEntries,
ndn::Face& face,
PartialProducer(ndn::Face& face,
ndn::KeyChain& keyChain,
size_t expectedNumEntries,
const ndn::Name& syncPrefix,
const ndn::Name& userPrefix,
ndn::time::milliseconds helloReplyFreshness = HELLO_REPLY_FRESHNESS,
@@ -76,7 +72,7 @@ public:
* @param seq the sequence number of the prefix
*/
void
publishName(const ndn::Name& prefix, ndn::optional<uint64_t> seq = ndn::nullopt);
publishName(const ndn::Name& prefix, std::optional<uint64_t> seq = std::nullopt);
private:
/**
+18 -18
View File
@@ -1,6 +1,6 @@
/* -*- Mode:C++; c-file-style:"gnu"; indent-tabs-mode:nil; -*- */
/*
* Copyright (c) 2014-2020, The University of Memphis
* Copyright (c) 2014-2023, The University of Memphis
*
* This file is part of PSync.
* See AUTHORS.md for complete list of PSync authors and contributors.
@@ -23,30 +23,29 @@
#include <ndn-cxx/util/exception.hpp>
#include <ndn-cxx/util/logger.hpp>
#include <cstring>
#include <limits>
namespace psync {
NDN_LOG_INIT(psync.ProducerBase);
ProducerBase::ProducerBase(size_t expectedNumEntries,
ndn::Face& face,
ProducerBase::ProducerBase(ndn::Face& face,
ndn::KeyChain& keyChain,
size_t expectedNumEntries,
const ndn::Name& syncPrefix,
const ndn::Name& userPrefix,
ndn::time::milliseconds syncReplyFreshness,
CompressionScheme ibltCompression,
CompressionScheme contentCompression)
: m_iblt(expectedNumEntries, ibltCompression)
: m_face(face)
, m_keyChain(keyChain)
, m_scheduler(m_face.getIoContext())
, m_rng(ndn::random::getRandomNumberEngine())
, m_iblt(expectedNumEntries, ibltCompression)
, m_segmentPublisher(m_face, m_keyChain)
, m_expectedNumEntries(expectedNumEntries)
, m_threshold(expectedNumEntries / 2)
, m_face(face)
, m_scheduler(m_face.getIoService())
, m_syncPrefix(syncPrefix)
, m_userPrefix(userPrefix)
, m_syncReplyFreshness(syncReplyFreshness)
, m_segmentPublisher(m_face, m_keyChain)
, m_rng(ndn::random::getRandomNumberEngine())
, m_ibltCompression(ibltCompression)
, m_contentCompression(contentCompression)
{
@@ -116,7 +115,7 @@ ProducerBase::updateSeqNo(const ndn::Name& prefix, uint64_t seq)
// Insert the new seq no in m_prefixes, m_biMap, and m_iblt
it->second = seq;
ndn::Name prefixWithSeq = ndn::Name(prefix).appendNumber(seq);
auto newHash = detail::murmurHash3(detail::N_HASHCHECK, prefixWithSeq.toUri());
auto newHash = detail::murmurHash3(detail::N_HASHCHECK, prefixWithSeq);
m_biMap.insert({newHash, prefixWithSeq});
m_iblt.insert(newHash);
}
@@ -125,22 +124,23 @@ void
ProducerBase::sendApplicationNack(const ndn::Name& name)
{
NDN_LOG_DEBUG("Sending application nack");
ndn::Name dataName(name);
m_iblt.appendToName(dataName);
dataName.appendSegment(0);
ndn::Data data(dataName);
data.setFreshnessPeriod(m_syncReplyFreshness);
data.setContentType(ndn::tlv::ContentType_Nack);
data.setFinalBlock(dataName[-1]);
data.setContentType(ndn::tlv::ContentType_Nack)
.setFreshnessPeriod(m_syncReplyFreshness)
.setFinalBlock(dataName[-1]);
m_keyChain.sign(data);
m_face.put(data);
}
void
ProducerBase::onRegisterFailed(const ndn::Name& prefix, const std::string& msg) const
ProducerBase::onRegisterFailed(const ndn::Name& prefix, const std::string& msg)
{
NDN_LOG_ERROR("ProducerBase::onRegisterFailed(" << prefix << "): " << msg);
NDN_LOG_ERROR("onRegisterFailed(" << prefix << "): " << msg);
NDN_THROW(Error(msg));
}
+30 -30
View File
@@ -1,6 +1,6 @@
/* -*- Mode:C++; c-file-style:"gnu"; indent-tabs-mode:nil; -*- */
/*
* Copyright (c) 2014-2021, The University of Memphis
* Copyright (c) 2014-2022, The University of Memphis
*
* This file is part of PSync.
* See AUTHORS.md for complete list of PSync authors and contributors.
@@ -27,7 +27,6 @@
#include <ndn-cxx/face.hpp>
#include <ndn-cxx/security/key-chain.hpp>
#include <ndn-cxx/security/validator-config.hpp>
#include <ndn-cxx/util/random.hpp>
#include <ndn-cxx/util/scheduler.hpp>
@@ -43,7 +42,7 @@ namespace bm = boost::bimaps;
/**
* @brief Base class for PartialProducer and FullProducer
*
* Contains code common to both
* Contains code common to both.
*/
class ProducerBase
{
@@ -56,35 +55,38 @@ public:
PSYNC_PUBLIC_WITH_TESTS_ELSE_PROTECTED:
/**
* @brief constructor
* @brief Constructor
*
* @param expectedNumEntries expected number entries in IBF
* @param face application's face
* @param face Application's face
* @param keyChain KeyChain instance to use for signing
* @param expectedNumEntries Expected number of entries in IBF
* @param syncPrefix The prefix of the sync group
* @param userPrefix The prefix of the first user in the group
* @param syncReplyFreshness freshness of sync data
* @param syncReplyFreshness FreshnessPeriod of sync data
* @param ibltCompression Compression scheme to use for IBF
* @param contentCompression Compression scheme to use for Data content
*/
ProducerBase(size_t expectedNumEntries,
ndn::Face& face,
ProducerBase(ndn::Face& face,
ndn::KeyChain& keyChain,
size_t expectedNumEntries,
const ndn::Name& syncPrefix,
const ndn::Name& userPrefix,
ndn::time::milliseconds syncReplyFreshness = SYNC_REPLY_FRESHNESS,
CompressionScheme ibltCompression = CompressionScheme::NONE,
CompressionScheme contentCompression = CompressionScheme::NONE);
public:
/**
* @brief Returns the current sequence number of the given prefix
*
* @param prefix prefix to get the sequence number of
*/
ndn::optional<uint64_t>
std::optional<uint64_t>
getSeqNo(const ndn::Name& prefix) const
{
auto it = m_prefixes.find(prefix);
if (it == m_prefixes.end()) {
return ndn::nullopt;
return std::nullopt;
}
return it->second;
}
@@ -146,17 +148,18 @@ PSYNC_PUBLIC_WITH_TESTS_ELSE_PROTECTED:
sendApplicationNack(const ndn::Name& name);
/**
* @brief Logs a message if setting an interest filter fails
* @brief Logs a message and throws if setting an interest filter fails
*/
void
onRegisterFailed(const ndn::Name& prefix, const std::string& msg) const;
[[noreturn]] static void
onRegisterFailed(const ndn::Name& prefix, const std::string& msg);
PSYNC_PUBLIC_WITH_TESTS_ELSE_PROTECTED:
ndn::Face& m_face;
ndn::KeyChain& m_keyChain;
ndn::Scheduler m_scheduler;
ndn::random::RandomNumberEngine& m_rng;
detail::IBLT m_iblt;
uint32_t m_expectedNumEntries;
// Threshold is used check if the differences are greater
// than it and whether we need to update the other side.
uint32_t m_threshold;
// prefix and sequence number
std::map<ndn::Name, uint64_t> m_prefixes;
@@ -165,20 +168,17 @@ PSYNC_PUBLIC_WITH_TESTS_ELSE_PROTECTED:
bm::unordered_set_of<ndn::Name, std::hash<ndn::Name>>>;
HashNameBiMap m_biMap;
ndn::Face& m_face;
ndn::KeyChain m_keyChain;
ndn::Scheduler m_scheduler;
ndn::Name m_syncPrefix;
ndn::Name m_userPrefix;
ndn::time::milliseconds m_syncReplyFreshness;
SegmentPublisher m_segmentPublisher;
ndn::random::RandomNumberEngine& m_rng;
CompressionScheme m_ibltCompression;
CompressionScheme m_contentCompression;
const size_t m_expectedNumEntries;
// Threshold is used check if the differences are greater
// than it and whether we need to update the other side.
const size_t m_threshold;
const ndn::Name m_syncPrefix;
const ndn::Name m_userPrefix;
const ndn::time::milliseconds m_syncReplyFreshness;
const CompressionScheme m_ibltCompression;
const CompressionScheme m_contentCompression;
};
} // namespace psync
+17 -61
View File
@@ -1,6 +1,6 @@
/* -*- Mode:C++; c-file-style:"gnu"; indent-tabs-mode:nil; -*- */
/*
* Copyright (c) 2014-2020, The University of Memphis
* Copyright (c) 2014-2023, The University of Memphis
*
* This file is part of PSync.
* See AUTHORS.md for complete list of PSync authors and contributors.
@@ -19,87 +19,43 @@
#include "PSync/segment-publisher.hpp"
#include <ndn-cxx/name-component.hpp>
namespace psync {
SegmentPublisher::SegmentPublisher(ndn::Face& face, ndn::KeyChain& keyChain, size_t imsLimit)
SegmentPublisher::SegmentPublisher(ndn::Face& face, ndn::KeyChain& keyChain,
const ndn::security::SigningInfo& signingInfo, size_t imsLimit)
: m_face(face)
, m_scheduler(m_face.getIoService())
, m_keyChain(keyChain)
, m_scheduler(m_face.getIoContext())
, m_segmenter(keyChain, signingInfo)
, m_ims(imsLimit)
{
}
void
SegmentPublisher::publish(const ndn::Name& interestName, const ndn::Name& dataName,
const ndn::Block& block, ndn::time::milliseconds freshness,
const ndn::security::SigningInfo& signingInfo)
ndn::span<const uint8_t> buffer, ndn::time::milliseconds freshness)
{
auto buf = std::make_shared<const ndn::Buffer>(block.wire(), block.size());
publish(interestName, dataName, buf, freshness, signingInfo);
}
auto segments = m_segmenter.segment(buffer, ndn::Name(dataName).appendVersion(),
ndn::MAX_NDN_PACKET_SIZE >> 1, freshness);
for (const auto& data : segments) {
m_ims.insert(*data, freshness);
m_scheduler.schedule(freshness, [this, name = data->getName()] { m_ims.erase(name); });
}
void
SegmentPublisher::publish(const ndn::Name& interestName, const ndn::Name& dataName,
const ndn::ConstBufferPtr& buffer,
ndn::time::milliseconds freshness,
const ndn::security::SigningInfo& signingInfo)
{
// Put on face only the segment which has a pending interest,
// otherwise the segment is unsolicited
uint64_t interestSegment = 0;
if (interestName[-1].isSegment()) {
interestSegment = interestName[-1].toSegment();
}
const uint8_t* rawBuffer = buffer->data();
const uint8_t* segmentBegin = rawBuffer;
const uint8_t* end = rawBuffer + buffer->size();
size_t maxPacketSize = (ndn::MAX_NDN_PACKET_SIZE >> 1);
uint64_t totalSegments = buffer->size() / maxPacketSize;
ndn::Name segmentPrefix(dataName);
segmentPrefix.appendVersion();
uint64_t segmentNo = 0;
do {
const uint8_t* segmentEnd = segmentBegin + maxPacketSize;
if (segmentEnd > end) {
segmentEnd = end;
}
ndn::Name segmentName(segmentPrefix);
segmentName.appendSegment(segmentNo);
// We get a std::exception: bad_weak_ptr from m_ims if we don't use shared_ptr for data
auto data = std::make_shared<ndn::Data>(segmentName);
data->setContent(segmentBegin, segmentEnd - segmentBegin);
data->setFreshnessPeriod(freshness);
data->setFinalBlock(ndn::name::Component::fromSegment(totalSegments));
segmentBegin = segmentEnd;
m_keyChain.sign(*data, signingInfo);
// Put on face only the segment which has a pending interest
// otherwise the segment is unsolicited
if (interestSegment == segmentNo) {
m_face.put(*data);
}
m_ims.insert(*data, freshness);
m_scheduler.schedule(freshness, [this, segmentName] { m_ims.erase(segmentName); });
++segmentNo;
} while (segmentBegin < end);
if (interestSegment < segments.size()) {
m_face.put(*segments[interestSegment]);
}
}
bool
SegmentPublisher::replyFromStore(const ndn::Name& interestName)
{
auto it = m_ims.find(interestName);
if (it != nullptr) {
m_face.put(*it);
return true;
+8 -25
View File
@@ -1,6 +1,6 @@
/* -*- Mode:C++; c-file-style:"gnu"; indent-tabs-mode:nil; -*- */
/*
* Copyright (c) 2014-2020, The University of Memphis
* Copyright (c) 2014-2023, The University of Memphis
*
* This file is part of PSync.
* See AUTHORS.md for complete list of PSync authors and contributors.
@@ -23,55 +23,38 @@
#include "PSync/detail/access-specifiers.hpp"
#include <ndn-cxx/face.hpp>
#include <ndn-cxx/name.hpp>
#include <ndn-cxx/ims/in-memory-storage-fifo.hpp>
#include <ndn-cxx/security/key-chain.hpp>
#include <ndn-cxx/util/scheduler.hpp>
#include <ndn-cxx/util/time.hpp>
#include <ndn-cxx/util/segmenter.hpp>
namespace psync {
/**
* @brief Segment Publisher to publish segmented data
* @brief Helper class to publish segmented data.
*/
class SegmentPublisher
{
public:
SegmentPublisher(ndn::Face& face, ndn::KeyChain& keyChain,
const ndn::security::SigningInfo& signingInfo = ndn::security::SigningInfo(),
size_t imsLimit = 100);
/**
* @brief Put all the segments in memory.
*
* @param interestName the interest name, to determine the sequence to be answered immediately
* @param dataName the data name, has components after interest name
* @param block the content of the data
* @param freshness freshness of the segments
* @param signingInfo signing info to sign the data with
*/
void
publish(const ndn::Name& interestName, const ndn::Name& dataName,
const ndn::Block& block, ndn::time::milliseconds freshness,
const ndn::security::SigningInfo& signingInfo = ndn::security::SigningInfo());
/**
* @brief Put all the segments in memory.
*
* @param interestName the interest name, to determine the sequence to be answered immediately
* @param dataName the data name, has components after interest name
* @param buffer the content of the data
* @param freshness freshness of the segments
* @param signingInfo signing info to sign the data with
* @param freshness freshness period of the segments
*/
void
publish(const ndn::Name& interestName, const ndn::Name& dataName,
const ndn::ConstBufferPtr& buffer, ndn::time::milliseconds freshness,
const ndn::security::SigningInfo& signingInfo = ndn::security::SigningInfo());
ndn::span<const uint8_t> buffer, ndn::time::milliseconds freshness);
/**
* @brief Try to reply from memory, return false if we cannot find the segment.
*
* The caller is then expected to use publish if this returns false.
* The caller is then expected to use publish() if this returns false.
*/
bool
replyFromStore(const ndn::Name& interestName);
@@ -79,7 +62,7 @@ public:
private:
ndn::Face& m_face;
ndn::Scheduler m_scheduler;
ndn::KeyChain& m_keyChain;
ndn::Segmenter m_segmenter;
PSYNC_PUBLIC_WITH_TESTS_ELSE_PRIVATE:
ndn::InMemoryStorageFifo m_ims;
+14 -14
View File
@@ -1,8 +1,9 @@
# PSync: Partial and Full Synchronization Library for NDN
![Language](https://img.shields.io/badge/C%2B%2B-14-blue.svg)
[![Build Status](https://travis-ci.org/named-data/PSync.svg?branch=master)](https://travis-ci.org/named-data/PSync)
![Latest Version](https://img.shields.io/github/tag/named-data/PSync.svg?color=darkkhaki&label=latest%20version)
[![CI](https://github.com/named-data/PSync/actions/workflows/ci.yml/badge.svg)](https://github.com/named-data/PSync/actions/workflows/ci.yml)
[![Docs](https://github.com/named-data/PSync/actions/workflows/docs.yml/badge.svg)](https://github.com/named-data/PSync/actions/workflows/docs.yml)
![Language](https://img.shields.io/badge/C%2B%2B-17-blue)
![Latest version](https://img.shields.io/github/v/tag/named-data/PSync?label=Latest%20version)
The PSync library implements the
[PSync protocol](https://named-data.net/wp-content/uploads/2017/05/scalable_name-based_data_synchronization.pdf).
@@ -18,7 +19,7 @@ PSync uses the [ndn-cxx](https://github.com/named-data/ndn-cxx) library.
### Prerequisites
* [ndn-cxx and its dependencies](https://named-data.net/doc/ndn-cxx/current/INSTALL.html)
* [ndn-cxx and its dependencies](https://docs.named-data.net/ndn-cxx/current/INSTALL.html)
### Build
@@ -28,11 +29,11 @@ To build PSync from source:
./waf
sudo ./waf install
To build on memory constrained platform, please use `./waf -j1` instead of `./waf`. The
command will disable parallel compilation.
To build on memory constrained systems, please use `./waf -j1` instead of `./waf`. This
will disable parallel compilation.
If configured with tests (`./waf configure --with-tests`), the above commands will also
generate unit tests that can be run with `./build/unit-tests`.
build a suite of unit tests that can be run with `./build/unit-tests`.
## Reporting bugs
@@ -41,13 +42,12 @@ Please submit any bug reports or feature requests to the
## Contributing
We greatly appreciate contributions to the PSync code base, provided that they are
licensed under the LGPL 3.0+ or a compatible license (see below).
If you are new to the NDN software community, please read the
[Contributor's Guide](https://github.com/named-data/.github/blob/master/CONTRIBUTING.md)
to get started.
Contributions to PSync are greatly appreciated and can be made through our
[Gerrit code review site](https://gerrit.named-data.net/).
If you are new to the NDN software community, please read our [Contributor's Guide](
https://github.com/named-data/.github/blob/main/CONTRIBUTING.md) to get started.
## License
PSync is an open source project licensed under the LGPL version 3.
See [`COPYING.md`](COPYING.md) for more information.
PSync is free software distributed under the GNU Lesser General Public License version 3.
See [`COPYING.md`](COPYING.md) for details.
-4
View File
@@ -1,4 +0,0 @@
PSync Release Notes
===================
.. include:: release-notes-latest.rst
-2
View File
@@ -1,2 +0,0 @@
*
!.gitignore
+26 -85
View File
@@ -1,32 +1,22 @@
# -*- coding: utf-8 -*-
#
# Configuration file for the Sphinx documentation builder.
#
# This file only contains a selection of the most common options. For a full
# list see the documentation:
# http://www.sphinx-doc.org/en/master/config
# For the full list of built-in configuration values, see the documentation:
# https://www.sphinx-doc.org/en/master/usage/configuration.html
# -- Path setup --------------------------------------------------------------
# If extensions (or modules to document with autodoc) are in another directory,
# add these directories to sys.path here. If the directory is relative to the
# documentation root, use os.path.abspath to make it absolute, like shown here.
#
# import os
import importlib.util
import sys
# sys.path.insert(0, os.path.abspath('.'))
# -- Project information -----------------------------------------------------
# https://www.sphinx-doc.org/en/master/usage/configuration.html#project-information
project = u'PSync: Full/Partial Synchronization Protocol for NDN'
copyright = u'Copyright © 2018-2020 Named Data Networking Project.'
author = u'Named Data Networking Project'
project = 'PSync: Partial/Full Synchronization Protocol for NDN'
copyright = 'Copyright © 2018-2023 Named Data Networking Project.'
author = 'Named Data Networking Project'
# The short X.Y version
# The short X.Y version.
#version = ''
# The full version, including alpha/beta/rc tags
# The full version, including alpha/beta/rc tags.
#release = ''
# There are two options for replacing |today|: either, you set today to some
@@ -37,49 +27,34 @@ today_fmt = '%Y-%m-%d'
# -- General configuration ---------------------------------------------------
# https://www.sphinx-doc.org/en/master/usage/configuration.html#general-configuration
# If your documentation needs a minimal Sphinx version, state it here.
#
needs_sphinx = '1.1'
# Add any Sphinx extension module names here, as strings. They can be
# extensions coming with Sphinx (named 'sphinx.ext.*') or your custom
# ones.
needs_sphinx = '4.0'
extensions = [
'sphinx.ext.extlinks',
'sphinx.ext.todo',
]
def addExtensionIfExists(extension):
def addExtensionIfExists(extension: str):
try:
__import__(extension)
if importlib.util.find_spec(extension) is None:
raise ModuleNotFoundError(extension)
except (ImportError, ValueError):
sys.stderr.write(f'WARNING: Extension {extension!r} not found. '
'Some documentation may not build correctly.\n')
else:
extensions.append(extension)
except ImportError:
sys.stderr.write("Extension '%s' not found. "
"Some documentation may not build correctly.\n" % extension)
addExtensionIfExists('sphinxcontrib.doxylink')
# The master toctree document.
master_doc = 'index'
# Add any paths that contain templates here, relative to this directory.
templates_path = ['_templates']
# List of patterns, relative to source directory, that match files and
# directories to ignore when looking for source files.
# This pattern also affects html_static_path and html_extra_path.
exclude_patterns = []
exclude_patterns = ['Thumbs.db', '.DS_Store']
# -- Options for HTML output -------------------------------------------------
# https://www.sphinx-doc.org/en/master/usage/configuration.html#options-for-html-output
# The theme to use for HTML and HTML Help pages. See the documentation for
# a list of builtin themes.
#
html_theme = 'named_data_theme'
# Add any paths that contain custom themes here, relative to this directory.
html_theme_path = ['.']
# Add any paths that contain custom static files (such as style sheets) here,
@@ -87,53 +62,19 @@ html_theme_path = ['.']
# so a file named "default.css" will overwrite the builtin "default.css".
html_static_path = ['_static']
html_copy_source = False
html_show_sourcelink = False
# -- Options for LaTeX output ------------------------------------------------
latex_elements = {
# The paper size ('letterpaper' or 'a4paper').
#
# 'papersize': 'letterpaper',
# The font size ('10pt', '11pt' or '12pt').
#
# 'pointsize': '10pt',
# Additional stuff for the LaTeX preamble.
#
# 'preamble': '',
# Latex figure (float) alignment
#
# 'figure_align': 'htbp',
}
# Grouping the document tree into LaTeX files. List of tuples
# (source start file, target name, title,
# author, documentclass [howto, manual, or own class]).
latex_documents = [
('index', 'PSync-docs.tex', u'Full/Partial Synchronization Protocol for NDN',
author, 'manual'),
]
# Disable syntax highlighting of code blocks by default.
highlight_language = 'none'
# -- Options for manual page output ------------------------------------------
# One entry per manual page. List of tuples
# (source start file, name, description, authors, manual section).
man_pages = [
]
# If true, show URL addresses after external links.
#man_show_urls = True
# -- Custom options ----------------------------------------------------------
# -- Misc options ------------------------------------------------------------
doxylink = {
'PSync': ('PSync.tag', 'doxygen/'),
}
extlinks = {
'issue': ('https://redmine.named-data.net/issues/%s', 'issue #'),
'issue': ('https://redmine.named-data.net/issues/%s', 'issue #%s'),
}
+14 -66
View File
@@ -68,7 +68,7 @@ OUTPUT_DIRECTORY = docs/doxygen
# performance problems for the file system.
# The default value is: NO.
CREATE_SUBDIRS = YES
CREATE_SUBDIRS = NO
# If the ALLOW_UNICODE_NAMES tag is set to YES, doxygen will allow non-ASCII
# characters to appear in the names of generated files. If set to NO, non-ASCII
@@ -152,7 +152,7 @@ FULL_PATH_NAMES = YES
# will be relative from the directory where doxygen is started.
# This tag requires that the tag FULL_PATH_NAMES is set to YES.
STRIP_FROM_PATH = ..
STRIP_FROM_PATH =
# The STRIP_FROM_INC_PATH tag can be used to strip a user-defined part of the
# path mentioned in the documentation of a class, which tells the reader which
@@ -161,7 +161,7 @@ STRIP_FROM_PATH = ..
# specify the list of include paths that are normally passed to the compiler
# using the -I flag.
STRIP_FROM_INC_PATH =
STRIP_FROM_INC_PATH = ..
# If the SHORT_NAMES tag is set to YES, doxygen will generate much shorter (but
# less readable) file names. This can be useful is your file systems doesn't
@@ -177,7 +177,7 @@ SHORT_NAMES = NO
# description.)
# The default value is: NO.
JAVADOC_AUTOBRIEF = NO
JAVADOC_AUTOBRIEF = YES
# If the QT_AUTOBRIEF tag is set to YES then doxygen will interpret the first
# line (until the first dot) of a Qt-style comment as the brief description. If
@@ -185,7 +185,7 @@ JAVADOC_AUTOBRIEF = NO
# requiring an explicit \brief command for a brief description.)
# The default value is: NO.
QT_AUTOBRIEF = NO
QT_AUTOBRIEF = YES
# The MULTILINE_CPP_IS_BRIEF tag can be set to YES to make doxygen treat a
# multi-line C++ special comment block (i.e. a block of //! or /// comments) as
@@ -230,12 +230,6 @@ TAB_SIZE = 4
ALIASES =
# This tag can be used to specify a number of word-keyword mappings (TCL only).
# A mapping has the form "name=value". For example adding "class=itcl::class"
# will allow you to use the command class in the itcl::class meaning.
TCL_SUBST =
# Set the OPTIMIZE_OUTPUT_FOR_C tag to YES if your project consists of C sources
# only. Doxygen will then generate output that is more tailored for C. For
# instance, some of the names that are used will be different. The list of all
@@ -443,7 +437,7 @@ EXTRACT_PACKAGE = NO
# included in the documentation.
# The default value is: NO.
EXTRACT_STATIC = YES
EXTRACT_STATIC = NO
# If the EXTRACT_LOCAL_CLASSES tag is set to YES, classes (and structs) defined
# locally in source files will be included in the documentation. If set to NO,
@@ -836,7 +830,7 @@ EXCLUDE_SYMLINKS = NO
# Note that the wildcards are matched against the file with absolute path, so to
# exclude all test directories for example use the pattern */test/*
EXCLUDE_PATTERNS =
EXCLUDE_PATTERNS = */PSync/impl/*
# The EXCLUDE_SYMBOLS tag can be used to specify one or more symbol names
# (namespaces, classes, functions, etc.) that should be excluded from the
@@ -1047,13 +1041,6 @@ VERBATIM_HEADERS = YES
ALPHABETICAL_INDEX = YES
# The COLS_IN_ALPHA_INDEX tag can be used to specify the number of columns in
# which the alphabetical index list will be split.
# Minimum value: 1, maximum value: 20, default value: 5.
# This tag requires that the tag ALPHABETICAL_INDEX is set to YES.
COLS_IN_ALPHA_INDEX = 5
# In case all classes in a project start with a common prefix, all classes will
# be put under the same header in the alphabetical index. The IGNORE_PREFIX tag
# can be used to specify a prefix (or a list of prefixes) that should be ignored
@@ -1191,7 +1178,7 @@ HTML_COLORSTYLE_GAMMA = 91
# The default value is: NO.
# This tag requires that the tag GENERATE_HTML is set to YES.
HTML_TIMESTAMP = YES
HTML_TIMESTAMP = NO
# If the HTML_DYNAMIC_SECTIONS tag is set to YES then the generated HTML
# documentation will contain sections that can be hidden and shown after the
@@ -1752,16 +1739,6 @@ LATEX_BATCHMODE = NO
LATEX_HIDE_INDICES = NO
# If the LATEX_SOURCE_CODE tag is set to YES then doxygen will include source
# code with syntax highlighting in the LaTeX output.
#
# Note that which sources are shown also depends on other settings such as
# SOURCE_BROWSER.
# The default value is: NO.
# This tag requires that the tag GENERATE_LATEX is set to YES.
LATEX_SOURCE_CODE = NO
# The LATEX_BIB_STYLE tag can be used to specify the style to use for the
# bibliography, e.g. plainnat, or ieeetr. See
# http://en.wikipedia.org/wiki/BibTeX and \cite for more info.
@@ -1834,16 +1811,6 @@ RTF_STYLESHEET_FILE =
RTF_EXTENSIONS_FILE =
# If the RTF_SOURCE_CODE tag is set to YES then doxygen will include source code
# with syntax highlighting in the RTF output.
#
# Note that which sources are shown also depends on other settings such as
# SOURCE_BROWSER.
# The default value is: NO.
# This tag requires that the tag GENERATE_RTF is set to YES.
RTF_SOURCE_CODE = NO
#---------------------------------------------------------------------------
# Configuration options related to the man page output
#---------------------------------------------------------------------------
@@ -1933,15 +1900,6 @@ GENERATE_DOCBOOK = NO
DOCBOOK_OUTPUT = docbook
# If the DOCBOOK_PROGRAMLISTING tag is set to YES, doxygen will include the
# program listings (including syntax highlighting and cross-referencing
# information) to the DOCBOOK output. Note that enabling this will significantly
# increase the size of the DOCBOOK output.
# The default value is: NO.
# This tag requires that the tag GENERATE_DOCBOOK is set to YES.
DOCBOOK_PROGRAMLISTING = NO
#---------------------------------------------------------------------------
# Configuration options for the AutoGen Definitions output
#---------------------------------------------------------------------------
@@ -2051,6 +2009,11 @@ INCLUDE_FILE_PATTERNS =
PREDEFINED = DOXYGEN \
NDN_CXX_DECLARE_WIRE_ENCODE_INSTANTIATIONS(x)= \
NDN_LOG_INIT(x)= \
NDN_LOG_MEMBER_DECL()= \
NDN_LOG_MEMBER_DECL_SPECIALIZED(x)= \
NDN_LOG_MEMBER_INIT(x,y)= \
NDN_LOG_MEMBER_INIT_SPECIALIZED(x,y)= \
PSYNC_PUBLIC_WITH_TESTS_ELSE_PROTECTED=protected \
PSYNC_PUBLIC_WITH_TESTS_ELSE_PRIVATE=private \
PSYNC_PROTECTED_WITH_TESTS_ELSE_PRIVATE=private \
@@ -2121,12 +2084,6 @@ EXTERNAL_GROUPS = YES
EXTERNAL_PAGES = YES
# The PERL_PATH should be the absolute path and name of the perl script
# interpreter (i.e. the result of 'which perl').
# The default file (with absolute path) is: /usr/bin/perl.
PERL_PATH = /usr/bin/perl
#---------------------------------------------------------------------------
# Configuration options related to the dot tool
#---------------------------------------------------------------------------
@@ -2140,15 +2097,6 @@ PERL_PATH = /usr/bin/perl
CLASS_DIAGRAMS = YES
# You can define message sequence charts within doxygen comments using the \msc
# command. Doxygen will then run the mscgen tool (see:
# http://www.mcternan.me.uk/mscgen/)) to produce the chart and insert it in the
# documentation. The MSCGEN_PATH tag allows you to specify the directory where
# the mscgen tool resides. If left empty the tool is assumed to be found in the
# default search path.
MSCGEN_PATH =
# You can include diagrams made with dia in doxygen documentation. Doxygen will
# then run dia to produce the diagram and insert it in the documentation. The
# DIA_PATH tag allows you to specify the directory where the dia binary resides.
@@ -2169,7 +2117,7 @@ HIDE_UNDOC_RELATIONS = YES
# set to NO
# The default value is: YES.
HAVE_DOT = YES
HAVE_DOT = @HAVE_DOT@
# The DOT_NUM_THREADS specifies the number of dot invocations doxygen is allowed
# to run in parallel. When set to 0 doxygen will base this on the number of
+87 -104
View File
@@ -1,158 +1,141 @@
PSync examples
==============
Examples
========
By default, examples in ``examples/`` folder are not built. To enable them, use
``--with-examples`` configure option. For example:
The example programs in the ``examples/`` subdirectory are not built by default.
To enable them, use the ``--with-examples`` configure option:
.. code-block:: bash
./waf configure --with-examples
./waf
Once built, the example binaries can be found under ``build/examples/``:
Example binary can be found under ``build/examples``\ :
* Full sync: ``psync-full-sync``
* Partial sync: ``psync-producer`` and ``psync-consumer``
* Full sync : ``psync-full-sync``
* Partial sync : ``psync-producer`` and ``psync-consumer``
If the library is installed to the system using ``./waf install`` then the examples
are also installed and can be executed directly.
If the library is installed to the system using ``./waf install``, then the examples
will also be installed and can be executed directly.
Partial Sync Example
--------------------
Partial sync example of PSync has two parts: producer and consumer.
These can be run on a machine after starting NFD:
The partial sync example of PSync has two parts: producer and consumer.
These can be run on a machine after starting NFD.
Producer
^^^^^^^^
* Enable the logs for producer:
* Enable logging for the producer.
.. code-block:: bash
.. code-block:: bash
export NDN_LOG=examples.PartialSyncProducerApp=INFO
export NDN_LOG="examples.PartialSyncProducerApp=INFO"
* Start the producer that will listen on ``/sync`` and publish one update for each
of ``/a-0`` ... ``/a-9``.
* Start the producer that will listen on /sync and publish 1 update for /a-0 ... /a-9 each.
.. code-block:: bash
.. code-block:: bash
psync-producer /sync /a 10 1
psync-producer /sync /a 10 1
* Sample output::
* Sample output:
.. code-block:: bash
1546280442.096296 INFO: [examples.PartialSyncProducerApp] Publish: /a-1/1
1546280456.053138 INFO: [examples.PartialSyncProducerApp] Publish: /a-6/1
1546280458.210415 INFO: [examples.PartialSyncProducerApp] Publish: /a-8/1
1546280469.954134 INFO: [examples.PartialSyncProducerApp] Publish: /a-5/1
1546280472.487425 INFO: [examples.PartialSyncProducerApp] Publish: /a-2/1
1546280473.466515 INFO: [examples.PartialSyncProducerApp] Publish: /a-7/1
1546280481.882258 INFO: [examples.PartialSyncProducerApp] Publish: /a-0/1
1546280484.201229 INFO: [examples.PartialSyncProducerApp] Publish: /a-4/1
1546280489.348968 INFO: [examples.PartialSyncProducerApp] Publish: /a-9/1
1546280491.420391 INFO: [examples.PartialSyncProducerApp] Publish: /a-3/1
1546280442.096296 INFO: [examples.PartialSyncProducerApp] Publish: /a-1/1
1546280456.053138 INFO: [examples.PartialSyncProducerApp] Publish: /a-6/1
1546280458.210415 INFO: [examples.PartialSyncProducerApp] Publish: /a-8/1
1546280469.954134 INFO: [examples.PartialSyncProducerApp] Publish: /a-5/1
1546280472.487425 INFO: [examples.PartialSyncProducerApp] Publish: /a-2/1
1546280473.466515 INFO: [examples.PartialSyncProducerApp] Publish: /a-7/1
1546280481.882258 INFO: [examples.PartialSyncProducerApp] Publish: /a-0/1
1546280484.201229 INFO: [examples.PartialSyncProducerApp] Publish: /a-4/1
1546280489.348968 INFO: [examples.PartialSyncProducerApp] Publish: /a-9/1
1546280491.420391 INFO: [examples.PartialSyncProducerApp] Publish: /a-3/1
Consumer
^^^^^^^^
* In a new terminal, enable the logs for consumer:
* In a new terminal, enable logging for the consumer.
.. code-block:: bash
.. code-block:: bash
export NDN_LOG=examples.PartialSyncConsumerApp=INFO
export NDN_LOG="examples.PartialSyncConsumerApp=INFO"
* Run the consumer to subscribe to 5 random prefixes from the publisher on ``/sync``.
* Run the consumer to subscribe to 5 random prefixes from the publisher on /sync
.. code-block:: bash
psync-consumer /sync 5
.. code-block:: bash
psync-consumer /sync 5
* Sample output from the consumer shows that it received updates only
for the subscribed prefixes:
for the subscribed prefixes::
.. code-block:: bash
1546280436.502769 INFO: [examples.PartialSyncConsumerApp] Subscribing to: /a-7
1546280436.502888 INFO: [examples.PartialSyncConsumerApp] Subscribing to: /a-9
1546280436.502911 INFO: [examples.PartialSyncConsumerApp] Subscribing to: /a-8
1546280436.502934 INFO: [examples.PartialSyncConsumerApp] Subscribing to: /a-4
1546280436.502956 INFO: [examples.PartialSyncConsumerApp] Subscribing to: /a-5
1546280458.211188 INFO: [examples.PartialSyncConsumerApp] Update: /a-8/1
1546280469.954886 INFO: [examples.PartialSyncConsumerApp] Update: /a-5/1
1546280473.467116 INFO: [examples.PartialSyncConsumerApp] Update: /a-7/1
1546280484.256181 INFO: [examples.PartialSyncConsumerApp] Update: /a-4/1
1546280489.349793 INFO: [examples.PartialSyncConsumerApp] Update: /a-9/1
1546280436.502769 INFO: [examples.PartialSyncConsumerApp] Subscribing to: /a-7
1546280436.502888 INFO: [examples.PartialSyncConsumerApp] Subscribing to: /a-9
1546280436.502911 INFO: [examples.PartialSyncConsumerApp] Subscribing to: /a-8
1546280436.502934 INFO: [examples.PartialSyncConsumerApp] Subscribing to: /a-4
1546280436.502956 INFO: [examples.PartialSyncConsumerApp] Subscribing to: /a-5
1546280458.211188 INFO: [examples.PartialSyncConsumerApp] Update: /a-8/1
1546280469.954886 INFO: [examples.PartialSyncConsumerApp] Update: /a-5/1
1546280473.467116 INFO: [examples.PartialSyncConsumerApp] Update: /a-7/1
1546280484.256181 INFO: [examples.PartialSyncConsumerApp] Update: /a-4/1
1546280489.349793 INFO: [examples.PartialSyncConsumerApp] Update: /a-9/1
Full Sync Example
-----------------
To demonstrate full sync mode of PSync, ``psync-full-sync``
can be run on a machine after starting NFD:
To demonstrate the full sync mode of PSync, ``psync-full-sync`` can be run on a
machine after starting NFD.
* Enable logging for the full sync demo app.
* Enable the logs for full sync:
.. code-block:: bash
.. code-block:: bash
export NDN_LOG="examples.FullSyncApp=INFO"
export NDN_LOG=examples.FullSyncApp=INFO
* Run the full sync example with sync prefix ``/sync``, user prefix ``/a``,
and publish three updates for each user prefix: ``/a-0`` and ``/a-1``.
This will simulate node *a*.
.. code-block:: bash
* Run the full sync example with sync prefix /sync, user prefix /a,
and publish three updates for each user prefix: /a-0 and /a-1. This will simulate node a.
psync-full-sync /sync /a 2 3
.. code-block:: bash
* Repeat for another user prefix, to simulate node *b*.
psync-full-sync /sync /a 2 3
.. code-block:: bash
psync-full-sync /sync /b 2 3
* Repeat for another user prefix, to simulate node b:
We should see that node *a* and node *b* have received each other's updates.
.. code-block:: bash
* Sample output from node *a* shows that it received all updates from node *b*
successfully::
psync-full-sync /sync /b 2 3
1546282730.759387 INFO: [examples.FullSyncApp] Update /b-1/1
1546282741.143225 INFO: [examples.FullSyncApp] Publish: /a-1/1
1546282749.375854 INFO: [examples.FullSyncApp] Publish: /a-0/1
1546282750.263246 INFO: [examples.FullSyncApp] Update /b-0/1
1546282765.875118 INFO: [examples.FullSyncApp] Update /b-1/2
1546282783.777807 INFO: [examples.FullSyncApp] Publish: /a-0/2
1546282794.565507 INFO: [examples.FullSyncApp] Publish: /a-0/3
1546282794.896895 INFO: [examples.FullSyncApp] Publish: /a-1/2
1546282803.839416 INFO: [examples.FullSyncApp] Update /b-0/2
1546282804.785867 INFO: [examples.FullSyncApp] Update /b-1/3
1546282845.273772 INFO: [examples.FullSyncApp] Publish: /a-1/3
1546282855.102790 INFO: [examples.FullSyncApp] Update /b-0/3
We should see that node a and node b have received each other's updates.
* Sample output from node *b*::
* Sample output from node a shows that it received all updates
from node b successfully:
.. code-block:: bash
1546282730.759387 INFO: [examples.FullSyncApp] Update /b-1/1
1546282741.143225 INFO: [examples.FullSyncApp] Publish: /a-1/1
1546282749.375854 INFO: [examples.FullSyncApp] Publish: /a-0/1
1546282750.263246 INFO: [examples.FullSyncApp] Update /b-0/1
1546282765.875118 INFO: [examples.FullSyncApp] Update /b-1/2
1546282783.777807 INFO: [examples.FullSyncApp] Publish: /a-0/2
1546282794.565507 INFO: [examples.FullSyncApp] Publish: /a-0/3
1546282794.896895 INFO: [examples.FullSyncApp] Publish: /a-1/2
1546282803.839416 INFO: [examples.FullSyncApp] Update /b-0/2
1546282804.785867 INFO: [examples.FullSyncApp] Update /b-1/3
1546282845.273772 INFO: [examples.FullSyncApp] Publish: /a-1/3
1546282855.102790 INFO: [examples.FullSyncApp] Update /b-0/3
* Sample output from node b:
.. code-block:: bash
1546282730.758296 INFO: [examples.FullSyncApp] Publish: /b-1/1
1546282741.144027 INFO: [examples.FullSyncApp] Update /a-1/1
1546282749.376543 INFO: [examples.FullSyncApp] Update /a-0/1
1546282750.262244 INFO: [examples.FullSyncApp] Publish: /b-0/1
1546282765.296005 INFO: [examples.FullSyncApp] Publish: /b-1/2
1546282783.778769 INFO: [examples.FullSyncApp] Update /a-0/2
1546282794.566485 INFO: [examples.FullSyncApp] Update /a-0/3
1546282795.374339 INFO: [examples.FullSyncApp] Update /a-1/2
1546282803.838394 INFO: [examples.FullSyncApp] Publish: /b-0/2
1546282804.033214 INFO: [examples.FullSyncApp] Publish: /b-1/3
1546282845.274680 INFO: [examples.FullSyncApp] Update /a-1/3
1546282855.101780 INFO: [examples.FullSyncApp] Publish: /b-0/3
1546282730.758296 INFO: [examples.FullSyncApp] Publish: /b-1/1
1546282741.144027 INFO: [examples.FullSyncApp] Update /a-1/1
1546282749.376543 INFO: [examples.FullSyncApp] Update /a-0/1
1546282750.262244 INFO: [examples.FullSyncApp] Publish: /b-0/1
1546282765.296005 INFO: [examples.FullSyncApp] Publish: /b-1/2
1546282783.778769 INFO: [examples.FullSyncApp] Update /a-0/2
1546282794.566485 INFO: [examples.FullSyncApp] Update /a-0/3
1546282795.374339 INFO: [examples.FullSyncApp] Update /a-1/2
1546282803.838394 INFO: [examples.FullSyncApp] Publish: /b-0/2
1546282804.033214 INFO: [examples.FullSyncApp] Publish: /b-1/3
1546282845.274680 INFO: [examples.FullSyncApp] Update /a-1/3
1546282855.101780 INFO: [examples.FullSyncApp] Publish: /b-0/3
+43 -28
View File
@@ -1,36 +1,51 @@
PSYNC - Partial/Full Sync Library based on BF and IBF
=====================================================
PSync is an ndn-cxx based C++ library for name synchronization that uses
PSync library implements the `PSync protocol <https://named-data.net/wp-content/uploads/2017/05/scalable_name-based_data_synchronization.pdf>`_. It uses Invertible
Bloom Lookup Table (IBLT), also known as Invertible Bloom Filter (IBF), to represent the state
of a producer in partial sync mode and the state of a node in full sync mode. An IBF is a compact data
structure where difference of two IBFs can be computed efficiently.
In partial sync, PSync uses a Bloom Filter to represent the subscription of list of the consumer.
PSync uses `ndn-cxx <https://github.com/named-data/ndn-cxx>`_ library as NDN development
library.
PSync is an open source project licensed under LGPL 3.0 (see ``COPYING.md`` for more
detail). We highly welcome all contributions to the PSync code base, provided that
they can be licensed under LGPL 3.0+ or other compatible license.
Please submit any bugs or issues to the `PSync issue tracker
<https://redmine.named-data.net/projects/PSync/issues>`__.
PSync Documentation
-------------------
PSync: Partial/Full Sync Library based on BF and IBF
====================================================
.. toctree::
:hidden:
:maxdepth: 3
:maxdepth: 2
install
examples
RELEASE-NOTES
release-notes
releases
- :doc:`install`
- :doc:`examples`
- :doc:`RELEASE-NOTES`
- :doc:`releases`
PSync is a C++ library for name synchronization that implements the `PSync protocol
<https://named-data.net/wp-content/uploads/2017/05/scalable_name-based_data_synchronization.pdf>`__.
It uses Invertible Bloom Lookup Table (IBLT), also known as Invertible Bloom Filter (IBF),
to represent the state of a producer in partial sync mode and the state of a node in full
sync mode. An IBF is a compact data structure where difference of two IBFs can be computed
efficiently. In partial sync, PSync uses a Bloom Filter to represent the subscription of
list of the consumer.
PSync uses the `ndn-cxx <https://github.com/named-data/ndn-cxx>`__ library.
Documentation
-------------
- :doc:`install`
- :doc:`examples`
- :doc:`release-notes`
- :doc:`releases`
Issues
------
Please submit any bug reports or feature requests to the
`PSync issue tracker <https://redmine.named-data.net/projects/psync/issues>`__.
Contributing
------------
Contributions to PSync are greatly appreciated and can be made through our
`Gerrit code review site <https://gerrit.named-data.net/>`__.
If you are new to the NDN software community, please read our `Contributor's Guide
<https://github.com/named-data/.github/blob/main/CONTRIBUTING.md>`__ to get started.
License
-------
PSync is free software: you can redistribute it and/or modify it under the terms of
the GNU Lesser General Public License as published by the Free Software Foundation,
either version 3 of the License, or (at your option) any later version. See `COPYING.md
<https://github.com/named-data/PSync/blob/master/COPYING.md>`__ for details.
+6 -9
View File
@@ -1,21 +1,18 @@
PSync Installation Instructions
===============================
.. toctree::
..
Installation Instructions
=========================
Prerequisites
-------------
- `ndn-cxx <https://named-data.net/doc/ndn-cxx>`_ and its dependencies
- `ndn-cxx <https://docs.named-data.net/ndn-cxx/>`__ and its dependencies
Build
-----
Execute the following commands to build PSync:
Run the following commands to build PSync:
::
.. code-block:: sh
./waf configure
./waf
sudo ./waf install
sudo ./waf install
+1 -1
View File
@@ -32,7 +32,7 @@ $mathjax
<!--top menu-->
<div class="nine columns" id="menu_container" >
<h1><a href="https://named-data.net/doc/PSync/$projectnumber/">$projectname $projectnumber documentation</a></h1>
<h1><a href="https://docs.named-data.net/PSync/$projectnumber/">$projectname $projectnumber documentation</a></h1>
</div>
</div>
</div><!--header container end-->
-1
View File
@@ -1 +0,0 @@
release-notes/release-notes-0.3.0.rst
+4
View File
@@ -0,0 +1,4 @@
Release Notes
=============
.. include:: release-notes/release-notes-0.4.0.rst
+9 -8
View File
@@ -1,14 +1,15 @@
PSync version 0.1.0
-------------------
Release date: January 25, 2019
*Release date: January 25, 2019*
Version 0.1.0 is the initial release of PSyncPSync library implements the `PSync protocol <https://named-data.net/wp-content/uploads/2017/05/scalable_name-based_data_synchronization.pdf>`_. It uses Invertible
Bloom Lookup Table (IBLT), also known as Invertible Bloom Filter (IBF), to represent the state
of a producer in partial sync mode and the state of a node in full sync mode. An IBF is a compact data
structure where difference of two IBFs can be computed efficiently.
Version 0.1.0 is the initial release of PSync. The PSync library implements the `PSync protocol
<https://named-data.net/wp-content/uploads/2017/05/scalable_name-based_data_synchronization.pdf>`__.
It uses Invertible Bloom Lookup Table (IBLT), also known as Invertible Bloom Filter (IBF), to represent
the state of a producer in partial sync mode and the state of a node in full sync mode. An IBF is a
compact data structure where difference of two IBFs can be computed efficiently.
In partial sync, PSync uses a Bloom Filter to represent the subscription of list of the consumer.
PSync uses `ndn-cxx <https://github.com/named-data/ndn-cxx>`_ library as NDN development
PSync uses the `ndn-cxx <https://github.com/named-data/ndn-cxx>`__ library as NDN development
library.
PSync is an open source project licensed under LGPL 3.0. We highly welcome all contributions to the PSync code base, provided that they can be licensed under LGPL 3.0+ or other compatible license.
@@ -29,6 +30,6 @@ PSync supports the following features:
- **Miscellaneous**
+ Uses ndn-cxx **SegmentFetcher** to segment hello and sync data.
+ Uses ndn-cxx ``SegmentFetcher`` to segment hello and sync data.
+ Provide a SegmentPublisher that can be used by other NDN applications.
+ Examples are provided in `examples` folder to show how to use the library.
+ Examples are provided in ``examples`` folder to show how to use the library.
+9 -5
View File
@@ -1,13 +1,17 @@
PSync version 0.2.0
-------------------
**New features**:
*Release date: February 2, 2020*
- **breaking change** Compress Sync Data in Full Sync by default (:issue:`5061`, :issue:`4917`)
New features
^^^^^^^^^^^^
- *(breaking change)* Compress Sync Data in Full Sync by default (:issue:`5061`, :issue:`4917`)
- Support various compression schemes for compressing IBF and Sync Data (:issue:`5061`)
**Changes**
Improvements and bug fixes
^^^^^^^^^^^^^^^^^^^^^^^^^^
- Use constant interest timeout for (discovery) sync Interest in Segment Fetcher (:issue:`4945`)
- Use constant interest timeout for (discovery) Sync Interest in ``SegmentFetcher`` (:issue:`4945`)
- Make default jitter shorter, react to NACK after jitter
- Use boost::bimap instead of two std::map variables to store hash to prefix matching
- Use ``boost::bimap`` instead of two ``std::map`` variables to store hash to prefix matching
+10 -6
View File
@@ -1,25 +1,29 @@
PSync version 0.3.0
-------------------
*Release date: January 3, 2021*
The build requirements have been increased to require Clang >= 4.0, Xcode >= 9.0, and Python >= 3.6.
Meanwhile, it is *recommended* to use GCC >= 7.4.0 and Boost >= 1.65.1.
This effectively drops official support for Ubuntu 16.04 when using distribution-provided Boost
packages -- PSync may still work on this platform, but we provide no official support for it.
We have taken some steps to be endian safe but PSync is not completely endian safe yet (:issue:`4818`)
New features
^^^^^^^^^^^^
- **breaking** Consumer: change hello data callback to include sequence number (:issue:`5122`)
- Support various compression schemes for compressing IBF and Sync Data (:issue:`5061`)
- *(breaking change)* Consumer: change hello data callback to include sequence number (:issue:`5122`)
Improvements and bug fixes
^^^^^^^^^^^^^^^^^^^^^^^^^^
- **breaking** IBLT: make encoding endian safe (:issue:`5076`)
- *(breaking change)* IBLT: make encoding endian safe (:issue:`5076`)
- Reset cached wire encoding after adding names (:issue:`5083`)
- Consumer reacts faster on sync Interest timeout (:issue:`5124`)
- Consumer reacts faster on Sync Interest timeout (:issue:`5124`)
- Move private classes and functions to ``psync::detail`` namespace
- Improved unit tests
Known issues
^^^^^^^^^^^^
- We have taken some steps to be endian safe but PSync is not completely endian safe yet
(:issue:`4818`)
@@ -0,0 +1,37 @@
PSync version 0.4.0
-------------------
*Release date: January 18, 2023*
Important changes and new features
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
- The minimum build requirements have been increased as follows:
- Either GCC >= 7.4.0 or Clang >= 6.0 is required on Linux
- On macOS, Xcode 11.3 or later is recommended; older versions may still work but are not
officially supported
- Boost >= 1.65.1 and ndn-cxx >= 0.8.1 are required on all platforms
- Sphinx 4.0 or later is required to build the documentation
- *(breaking change)* Switch to C++17
- *(breaking change)* The Name TLV value is now hashed directly instead of being converted
to URI format first (:issue:`4838`)
- Add ``incomingFace`` field to missing data notifications (:issue:`3626`)
- *(breaking change)* Add ``ndn::KeyChain`` parameter to the producer API
- Provide API to remove a subscription in partial sync :psync:`Consumer` (:issue:`5242`)
Improvements and bug fixes
^^^^^^^^^^^^^^^^^^^^^^^^^^
- Use ndn-cxx's ``ndn::util::Segmenter`` class in :psync:`SegmentPublisher`
- Fix compilation against the latest version of ndn-cxx
- Stop using the ``gold`` linker on Linux; prefer instead linking with ``lld`` if installed
- Update waf build system to version 2.0.24
- Various documentation improvements
Known issues
^^^^^^^^^^^^
- We have taken some steps to be endian safe but PSync is not completely endian safe yet
(:issue:`4818`)
+4 -5
View File
@@ -1,10 +1,9 @@
PSync Versions
++++++++++++++
==============
.. toctree::
:hidden:
:glob:
:maxdepth: 1
:reversed:
release-notes/release-notes-0.3.0
release-notes/release-notes-0.2.0
release-notes/release-notes-0.1.0
release-notes/*
+13 -14
View File
@@ -1,6 +1,6 @@
/* -*- Mode:C++; c-file-style:"gnu"; indent-tabs-mode:nil; -*- */
/*
* Copyright (c) 2014-2020, The University of Memphis
* Copyright (c) 2014-2023, The University of Memphis
*
* This file is part of PSync.
* See AUTHORS.md for complete list of PSync authors and contributors.
@@ -19,7 +19,7 @@
#include <PSync/consumer.hpp>
#include <ndn-cxx/name.hpp>
#include <ndn-cxx/face.hpp>
#include <ndn-cxx/util/logger.hpp>
#include <ndn-cxx/util/random.hpp>
@@ -32,19 +32,18 @@ class PSyncConsumer
public:
/**
* @brief Initialize consumer and start hello process
*
* 0.001 is the false positive probability of the bloom filter
*
* @param syncPrefix should be the same as producer
* @param nSub number of subscriptions is used for the bloom filter (subscription list) size
*/
PSyncConsumer(const ndn::Name& syncPrefix, int nSub)
: m_nSub(nSub)
, m_consumer(syncPrefix, m_face,
std::bind(&PSyncConsumer::afterReceiveHelloData, this, _1),
std::bind(&PSyncConsumer::processSyncUpdate, this, _1),
m_nSub, 0.001)
, m_rng(ndn::random::getRandomNumberEngine())
, m_consumer(m_face, syncPrefix, [this] {
psync::Consumer::Options opts;
opts.onHelloData = std::bind(&PSyncConsumer::afterReceiveHelloData, this, _1);
opts.onUpdate = std::bind(&PSyncConsumer::processSyncUpdate, this, _1);
opts.bfCount = m_nSub;
return opts;
} ())
{
// This starts the consumer side by sending a hello interest to the producer
// When the producer responds with hello data, afterReceiveHelloData is called
@@ -96,18 +95,18 @@ private:
private:
ndn::Face m_face;
int m_nSub;
int m_nSub;
psync::Consumer m_consumer;
ndn::random::RandomNumberEngine& m_rng;
ndn::random::RandomNumberEngine& m_rng{ndn::random::getRandomNumberEngine()};
};
int
main(int argc, char* argv[])
{
if (argc != 3) {
std::cout << "usage: " << argv[0] << " "
<< "<sync-prefix> <number-of-subscriptions>" << std::endl;
std::cerr << "Usage: " << argv[0] << " <sync-prefix> <number-of-subscriptions>\n";
return 1;
}
+19 -24
View File
@@ -1,6 +1,6 @@
/* -*- Mode:C++; c-file-style:"gnu"; indent-tabs-mode:nil; -*- */
/*
* Copyright (c) 2014-2019, The University of Memphis
* Copyright (c) 2014-2023, The University of Memphis
*
* This file is part of PSync.
* See AUTHORS.md for complete list of PSync authors and contributors.
@@ -20,6 +20,7 @@
#include <PSync/full-producer.hpp>
#include <ndn-cxx/face.hpp>
#include <ndn-cxx/security/key-chain.hpp>
#include <ndn-cxx/util/logger.hpp>
#include <ndn-cxx/util/random.hpp>
#include <ndn-cxx/util/scheduler.hpp>
@@ -42,20 +43,16 @@ public:
*/
Producer(const ndn::Name& syncPrefix, const std::string& userPrefix,
int numDataStreams, int maxNumPublish)
: m_scheduler(m_face.getIoService())
, m_fullProducer(80, m_face, syncPrefix, userPrefix,
std::bind(&Producer::processSyncUpdate, this, _1),
1600_ms, 1600_ms)
, m_numDataStreams(numDataStreams)
: m_producer(m_face, m_keyChain, 80, syncPrefix, userPrefix,
std::bind(&Producer::processSyncUpdate, this, _1),
1600_ms, 1600_ms)
, m_maxNumPublish(maxNumPublish)
, m_rng(ndn::random::getRandomNumberEngine())
, m_rangeUniformRandom(0, 60000)
{
// Add user prefixes and schedule updates for them in specified interval
for (int i = 0; i < m_numDataStreams; i++) {
ndn::Name prefix(userPrefix + "-" + ndn::to_string(i));
m_fullProducer.addUserNode(prefix);
m_scheduler.schedule(ndn::time::milliseconds(m_rangeUniformRandom(m_rng)),
for (int i = 0; i < numDataStreams; i++) {
ndn::Name prefix(userPrefix + "-" + std::to_string(i));
m_producer.addUserNode(prefix);
m_scheduler.schedule(ndn::time::milliseconds(m_uniformRand(m_rng)),
[this, prefix] { doUpdate(prefix); });
}
}
@@ -70,13 +67,13 @@ private:
void
doUpdate(const ndn::Name& prefix)
{
m_fullProducer.publishName(prefix);
m_producer.publishName(prefix);
uint64_t seqNo = m_fullProducer.getSeqNo(prefix).value();
uint64_t seqNo = m_producer.getSeqNo(prefix).value();
NDN_LOG_INFO("Publish: " << prefix << "/" << seqNo);
if (seqNo < m_maxNumPublish) {
m_scheduler.schedule(ndn::time::milliseconds(m_rangeUniformRandom(m_rng)),
m_scheduler.schedule(ndn::time::milliseconds(m_uniformRand(m_rng)),
[this, prefix] { doUpdate(prefix); });
}
}
@@ -93,24 +90,22 @@ private:
private:
ndn::Face m_face;
ndn::Scheduler m_scheduler;
ndn::KeyChain m_keyChain;
ndn::Scheduler m_scheduler{m_face.getIoContext()};
psync::FullProducer m_fullProducer;
int m_numDataStreams;
psync::FullProducer m_producer;
uint64_t m_maxNumPublish;
ndn::random::RandomNumberEngine& m_rng;
std::uniform_int_distribution<> m_rangeUniformRandom;
ndn::random::RandomNumberEngine& m_rng{ndn::random::getRandomNumberEngine()};
std::uniform_int_distribution<> m_uniformRand{0, 60000};
};
int
main(int argc, char* argv[])
{
if (argc != 5) {
std::cout << "usage: " << argv[0] << " <syncPrefix> <user-prefix> "
<< "<number-of-user-prefixes> <max-number-of-updates-per-user-prefix>"
<< std::endl;
std::cerr << "Usage: " << argv[0] << " <sync-prefix> <user-prefix> "
<< "<number-of-user-prefixes> <max-number-of-updates-per-user-prefix>\n";
return 1;
}
+16 -21
View File
@@ -1,6 +1,6 @@
/* -*- Mode:C++; c-file-style:"gnu"; indent-tabs-mode:nil; -*- */
/*
* Copyright (c) 2014-2019, The University of Memphis
* Copyright (c) 2014-2023, The University of Memphis
*
* This file is part of PSync.
* See AUTHORS.md for complete list of PSync authors and contributors.
@@ -20,6 +20,7 @@
#include <PSync/partial-producer.hpp>
#include <ndn-cxx/face.hpp>
#include <ndn-cxx/security/key-chain.hpp>
#include <ndn-cxx/util/logger.hpp>
#include <ndn-cxx/util/random.hpp>
#include <ndn-cxx/util/scheduler.hpp>
@@ -37,24 +38,20 @@ public:
* IBF size is set to 40 in m_producer as the expected number of update to IBF in a sync cycle
*/
PSyncPartialProducer(const ndn::Name& syncPrefix, const std::string& userPrefix,
int nDataStreams, int maxNumPublish)
: m_scheduler(m_face.getIoService())
, m_producer(40, m_face, syncPrefix, userPrefix + "-0")
, m_nDataStreams(nDataStreams)
int numDataStreams, int maxNumPublish)
: m_producer(m_face, m_keyChain, 40, syncPrefix, userPrefix + "-0")
, m_maxNumPublish(maxNumPublish)
, m_rng(ndn::random::getRandomNumberEngine())
, m_rangeUniformRandom(0, 60000)
{
// Add user prefixes and schedule updates for them
for (int i = 0; i < m_nDataStreams; i++) {
ndn::Name updateName(userPrefix + "-" + ndn::to_string(i));
for (int i = 0; i < numDataStreams; i++) {
ndn::Name updateName(userPrefix + "-" + std::to_string(i));
// Add the user prefix to the producer
// Note that this does not add the already added userPrefix-0 in the constructor
m_producer.addUserNode(updateName);
// Each user prefix is updated at random interval between 0 and 60 second
m_scheduler.schedule(ndn::time::milliseconds(m_rangeUniformRandom(m_rng)),
// Each user prefix is updated at a random interval between 0 and 60 seconds
m_scheduler.schedule(ndn::time::milliseconds(m_uniformRand(m_rng)),
[this, updateName] { doUpdate(updateName); });
}
}
@@ -76,32 +73,30 @@ private:
NDN_LOG_INFO("Publish: " << updateName << "/" << seqNo);
if (seqNo < m_maxNumPublish) {
// Schedule the next update for this user prefix b/w 0 and 60 seconds
m_scheduler.schedule(ndn::time::milliseconds(m_rangeUniformRandom(m_rng)),
// Schedule the next update for this user prefix between 0 and 60 seconds
m_scheduler.schedule(ndn::time::milliseconds(m_uniformRand(m_rng)),
[this, updateName] { doUpdate(updateName); });
}
}
private:
ndn::Face m_face;
ndn::Scheduler m_scheduler;
ndn::KeyChain m_keyChain;
ndn::Scheduler m_scheduler{m_face.getIoContext()};
psync::PartialProducer m_producer;
int m_nDataStreams;
uint64_t m_maxNumPublish;
ndn::random::RandomNumberEngine& m_rng;
std::uniform_int_distribution<int> m_rangeUniformRandom;
ndn::random::RandomNumberEngine& m_rng{ndn::random::getRandomNumberEngine()};
std::uniform_int_distribution<> m_uniformRand{0, 60000};
};
int
main(int argc, char* argv[])
{
if (argc != 5) {
std::cout << "usage: " << argv[0] << " <sync-prefix> <user-prefix> "
<< "<number-of-user-prefixes> <max-number-of-updates-per-user-prefix>"
<< std::endl;
std::cerr << "Usage: " << argv[0] << " <sync-prefix> <user-prefix> "
<< "<number-of-user-prefixes> <max-number-of-updates-per-user-prefix>\n";
return 1;
}
+3 -3
View File
@@ -6,7 +6,7 @@ def build(bld):
# List all .cpp files (whole example in one .cpp)
for ex in bld.path.ant_glob('*.cpp'):
name = ex.change_ext('').path_from(bld.path.get_bld())
bld.program(name='example-%s' % name,
target='psync-%s' % name,
bld.program(name=f'example-{name}',
target=f'psync-{name}',
source=[ex],
use='PSync')
use='PSync')
+5 -7
View File
@@ -1,6 +1,6 @@
/* -*- Mode: C++; c-file-style: "gnu"; indent-tabs-mode:nil -*- */
/*
* Copyright (c) 2013-2020 Regents of the University of California
* Copyright (c) 2013-2022 Regents of the University of California
* The University of Memphis
*
* This file is part of PSync.
@@ -19,12 +19,11 @@
#include "tests/clock-fixture.hpp"
namespace ndn {
namespace tests {
namespace psync::tests {
ClockFixture::ClockFixture()
: m_steadyClock(make_shared<time::UnitTestSteadyClock>())
, m_systemClock(make_shared<time::UnitTestSystemClock>())
: m_steadyClock(std::make_shared<time::UnitTestSteadyClock>())
, m_systemClock(std::make_shared<time::UnitTestSystemClock>())
{
time::setCustomClocks(m_steadyClock, m_systemClock);
}
@@ -50,5 +49,4 @@ ClockFixture::advanceClocks(time::nanoseconds tick, time::nanoseconds total)
}
}
} // namespace tests
} // namespace ndn
} // namespace psync::tests
+7 -7
View File
@@ -1,6 +1,6 @@
/* -*- Mode: C++; c-file-style: "gnu"; indent-tabs-mode:nil -*- */
/*
* Copyright (c) 2013-2020 Regents of the University of California
* Copyright (c) 2013-2022 Regents of the University of California
* The University of Memphis
*
* This file is part of PSync.
@@ -22,8 +22,9 @@
#include <ndn-cxx/util/time-unit-test-clock.hpp>
namespace ndn {
namespace tests {
namespace psync::tests {
namespace time = ndn::time;
/** \brief A test fixture that overrides steady clock and system clock.
*/
@@ -73,11 +74,10 @@ private:
}
protected:
shared_ptr<time::UnitTestSteadyClock> m_steadyClock;
shared_ptr<time::UnitTestSystemClock> m_systemClock;
std::shared_ptr<time::UnitTestSteadyClock> m_steadyClock;
std::shared_ptr<time::UnitTestSystemClock> m_systemClock;
};
} // namespace tests
} // namespace ndn
} // namespace psync::tests
#endif // PSYNC_TESTS_CLOCK_FIXTURE_HPP
+6 -12
View File
@@ -1,7 +1,7 @@
/* -*- Mode: C++; c-file-style: "gnu"; indent-tabs-mode:nil -*- */
/*
* Copyright (c) 2013-2020 Regents of the University of California
* The University of Memphis
* Copyright (c) 2013-2023, Regents of the University of California,
* The University of Memphis.
*
* This file is part of PSync.
*
@@ -22,10 +22,9 @@
#include "tests/clock-fixture.hpp"
#include <boost/asio/io_service.hpp>
#include <boost/asio/io_context.hpp>
namespace ndn {
namespace tests {
namespace psync::tests {
class IoFixture : public ClockFixture
{
@@ -34,20 +33,15 @@ private:
afterTick() final
{
if (m_io.stopped()) {
#if BOOST_VERSION >= 106600
m_io.restart();
#else
m_io.reset();
#endif
}
m_io.poll();
}
protected:
boost::asio::io_service m_io;
boost::asio::io_context m_io;
};
} // namespace tests
} // namespace ndn
} // namespace psync::tests
#endif // PSYNC_TESTS_IO_FIXTURE_HPP
+37
View File
@@ -0,0 +1,37 @@
/* -*- Mode:C++; c-file-style:"gnu"; indent-tabs-mode:nil; -*- */
/*
* Copyright (c) 2013-2022 Regents of the University of California.
*
* This file is part of PSync.
*
* PSync is free software: you can redistribute it and/or modify it under the terms
* of the GNU Lesser General Public License as published by the Free Software Foundation, either
* version 3 of the License, or (at your option) any later version.
*
* PSync is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY;
* without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR
* PURPOSE. See the GNU Lesser General Public License for more details.
*
* You should have received a copy of the GNU Lesser General Public License along with
* PSync, e.g., in COPYING.md file. If not, see <http://www.gnu.org/licenses/>.
*/
#ifndef PSYNC_TESTS_KEY_CHAIN_FIXTURE_HPP
#define PSYNC_TESTS_KEY_CHAIN_FIXTURE_HPP
#include <ndn-cxx/security/key-chain.hpp>
namespace psync::tests {
/**
* @brief A fixture providing an in-memory KeyChain.
*/
class KeyChainFixture
{
protected:
ndn::KeyChain m_keyChain{"pib-memory:", "tpm-memory:"};
};
} // namespace psync::tests
#endif // PSYNC_TESTS_KEY_CHAIN_FIXTURE_HPP
+28 -22
View File
@@ -1,6 +1,6 @@
/* -*- Mode:C++; c-file-style:"gnu"; indent-tabs-mode:nil; -*- */
/*
* Copyright (c) 2014-2020, The University of Memphis
* Copyright (c) 2014-2023, The University of Memphis
*
* This file is part of PSync.
* See AUTHORS.md for complete list of PSync authors and contributors.
@@ -22,31 +22,20 @@
#include "tests/boost-test.hpp"
#include "tests/io-fixture.hpp"
#include <ndn-cxx/name.hpp>
#include <ndn-cxx/util/dummy-client-face.hpp>
namespace psync {
using namespace ndn;
using ndn::Name;
BOOST_AUTO_TEST_SUITE(TestConsumer)
BOOST_AUTO_TEST_CASE(Constructor)
{
util::DummyClientFace face({true, true});
BOOST_REQUIRE_NO_THROW(Consumer(Name("/psync"), face,
[] (const auto&) {},
[] (const auto&) {},
40, 0.001));
}
BOOST_AUTO_TEST_CASE(AddSubscription)
{
util::DummyClientFace face({true, true});
Consumer consumer(Name("/psync"), face,
[] (const auto&) {},
[] (const auto&) {},
40, 0.001);
ndn::DummyClientFace face;
Consumer::Options opts;
opts.bfCount = 40;
Consumer consumer(face, "/psync", opts);
Name subscription("test");
@@ -55,13 +44,30 @@ BOOST_AUTO_TEST_CASE(AddSubscription)
BOOST_CHECK(!consumer.addSubscription(subscription, 0));
}
BOOST_AUTO_TEST_CASE(RemoveSubscription)
{
ndn::DummyClientFace face;
Consumer::Options opts;
opts.bfCount = 40;
Consumer consumer(face, "/psync", opts);
Name subscription("test");
consumer.addSubscription(subscription, 0);
BOOST_CHECK(consumer.isSubscribed(subscription));
BOOST_CHECK(consumer.removeSubscription(subscription));
BOOST_CHECK(!consumer.removeSubscription(subscription));
BOOST_CHECK(!consumer.isSubscribed(subscription));
}
BOOST_FIXTURE_TEST_CASE(ConstantTimeoutForFirstSegment, tests::IoFixture)
{
util::DummyClientFace face(m_io, {true, true});
Consumer consumer(Name("/psync"), face,
[] (const auto&) {},
[] (const auto&) {},
40, 0.001, 4_s, 4_s);
ndn::DummyClientFace face(m_io);
Consumer::Options opts;
opts.bfCount = 40;
opts.helloInterestLifetime = 4_s;
opts.syncInterestLifetime = 4_s;
Consumer consumer(face, "/psync", opts);
consumer.sendHelloInterest();
advanceClocks(4_s);
+19 -23
View File
@@ -1,6 +1,6 @@
/* -*- Mode:C++; c-file-style:"gnu"; indent-tabs-mode:nil; -*- */
/*
* Copyright (c) 2014-2020, The University of Memphis
* Copyright (c) 2014-2023, The University of Memphis
*
* This file is part of PSync.
* See AUTHORS.md for complete list of PSync authors and contributors.
@@ -22,65 +22,61 @@
#include "tests/boost-test.hpp"
#include "tests/io-fixture.hpp"
#include "tests/key-chain-fixture.hpp"
#include <ndn-cxx/mgmt/nfd/control-parameters.hpp>
#include <ndn-cxx/util/dummy-client-face.hpp>
namespace psync {
using namespace ndn;
using ndn::Interest;
using ndn::Name;
BOOST_AUTO_TEST_SUITE(TestFullProducer)
BOOST_AUTO_TEST_CASE(Constructor)
class FullProducerFixture : public tests::IoFixture, public tests::KeyChainFixture
{
util::DummyClientFace face({true, true});
BOOST_CHECK_NO_THROW(FullProducer(40, face, Name("/psync"), Name("/testUser"), nullptr));
}
protected:
ndn::DummyClientFace m_face{m_io, m_keyChain, {true, true}};
};
BOOST_FIXTURE_TEST_SUITE(TestFullProducer, FullProducerFixture)
BOOST_AUTO_TEST_CASE(OnInterest)
{
Name syncPrefix("/psync"), userNode("/testUser");
util::DummyClientFace face({true, true});
FullProducer node(40, face, syncPrefix, userNode, nullptr);
FullProducer node(m_face, m_keyChain, 40, syncPrefix, userNode, nullptr);
Name syncInterestName(syncPrefix);
syncInterestName.append("malicious-IBF");
BOOST_REQUIRE_NO_THROW(node.onSyncInterest(syncPrefix, Interest(syncInterestName)));
BOOST_CHECK_NO_THROW(node.onSyncInterest(syncPrefix, Interest(syncInterestName)));
}
BOOST_FIXTURE_TEST_CASE(ConstantTimeoutForFirstSegment, tests::IoFixture)
BOOST_AUTO_TEST_CASE(ConstantTimeoutForFirstSegment)
{
Name syncPrefix("/psync"), userNode("/testUser");
util::DummyClientFace face(m_io, {true, true});
FullProducer node(m_face, m_keyChain, 40, syncPrefix, userNode, nullptr, 8_s);
FullProducer node(40, face, syncPrefix, userNode, nullptr, 8_s);
advanceClocks(10_ms);
face.sentInterests.clear();
m_face.sentInterests.clear();
// full sync sends the next one in interest lifetime / 2 +- jitter
advanceClocks(6_s);
BOOST_CHECK_EQUAL(face.sentInterests.size(), 1);
BOOST_CHECK_EQUAL(m_face.sentInterests.size(), 1);
}
BOOST_AUTO_TEST_CASE(OnSyncDataDecodeFailure)
{
Name syncPrefix("/psync"), userNode("/testUser");
util::DummyClientFace face({true, true});
FullProducer node(40, face, syncPrefix, userNode, nullptr);
FullProducer node(m_face, m_keyChain, 40, syncPrefix, userNode, nullptr);
Name syncInterestName(syncPrefix);
node.m_iblt.appendToName(syncInterestName);
Interest syncInterest(syncInterestName);
auto badCompress = std::make_shared<const Buffer>(5);
auto badCompress = std::make_shared<const ndn::Buffer>(5);
BOOST_CHECK_NO_THROW(node.onSyncData(syncInterest, badCompress));
const uint8_t test[] = {'t', 'e', 's', 't'};
auto goodCompressBadBlock = detail::compress(node.m_contentCompression, test, sizeof(test));
auto goodCompressBadBlock = detail::compress(node.m_contentCompression, test);
BOOST_CHECK_NO_THROW(node.onSyncData(syncInterest, goodCompressBadBlock));
}
+253 -176
View File
@@ -1,6 +1,6 @@
/* -*- Mode:C++; c-file-style:"gnu"; indent-tabs-mode:nil; -*- */
/*
* Copyright (c) 2014-2020, The University of Memphis
* Copyright (c) 2014-2023, The University of Memphis
*
* This file is part of PSync.
* See AUTHORS.md for complete list of PSync authors and contributors.
@@ -24,32 +24,160 @@
#include "tests/boost-test.hpp"
#include "tests/io-fixture.hpp"
#include "tests/key-chain-fixture.hpp"
#include <array>
#include <ndn-cxx/util/dummy-client-face.hpp>
namespace psync {
using namespace ndn;
using ndn::Interest;
using ndn::Name;
class FullSyncFixture : public tests::IoFixture
class FullSyncFixture : public tests::IoFixture, public tests::KeyChainFixture
{
protected:
void
addNode(int id)
{
BOOST_ASSERT(id >= 0 && id <= 3);
BOOST_ASSERT(id >= 0 && id < MAX_NODES);
userPrefixes[id] = "/userPrefix" + std::to_string(id);
faces[id] = std::make_unique<ndn::DummyClientFace>(m_io, m_keyChain,
ndn::DummyClientFace::Options{true, true});
nodes[id] = std::make_unique<FullProducer>(*faces[id], m_keyChain, 40, syncPrefix, userPrefixes[id],
[] (const auto&) {});
}
faces[id] = std::make_shared<util::DummyClientFace>(m_io, util::DummyClientFace::Options{true, true});
userPrefixes[id] = Name("userPrefix" + to_string(id));
nodes[id] = make_shared<FullProducer>(40, *faces[id], syncPrefix, userPrefixes[id],
[] (const auto&) {});
void
clearNodes()
{
nodes = {};
faces = {};
userPrefixes = {};
}
/**
* @brief Return a user prefix in the form /userNode<id>-<i>.
* @param id update originator node index.
* @param i user prefix index.
*/
static Name
makeSubPrefix(int id, int i)
{
return "/userNode" + std::to_string(id) + "-" + std::to_string(i);
}
/**
* @brief Publish a batch of updates.
* @param id node index.
* @param min minimum user prefix index.
* @param min maximum user prefix index.
* @param seq update sequence number.
* @post nodes[id] has user nodes /userNode<id>-<i> i[min,max] , with sequence number
* set to @p seq ; only one sync Data may be sent after the last update.
*/
void
batchUpdate(int id, int min, int max, uint64_t seq)
{
FullProducer& node = *nodes.at(id);
for (int i = min; i <= max; i++) {
auto userPrefix = makeSubPrefix(id, i);
node.addUserNode(userPrefix);
if (i < max) {
node.updateSeqNo(userPrefix, seq);
}
else {
node.publishName(userPrefix, seq);
}
}
}
/**
* @brief Check sequence number on a batch of user prefixes.
* @param id node index where the check is performed.
* @param origin update originator node index for deriving user prefixes.
* @param min minimum user prefix index.
* @param max maximum user prefix index.
* @param seq expected sequence number.
*/
void
batchCheck(int id, int origin, int min, int max, std::optional<uint64_t> seq)
{
uint64_t expected = seq.value_or(NOT_EXIST);
FullProducer& node = *nodes.at(id);
for (int i = min; i <= max; i++) {
auto userPrefix = makeSubPrefix(origin, i);
BOOST_TEST_CONTEXT("node=" << id << " userPrefix=" << userPrefix) {
BOOST_CHECK_EQUAL(node.getSeqNo(userPrefix).value_or(NOT_EXIST), expected);
}
}
}
struct IbfDecodeFailureCounts
{
size_t aboveThreshold = 0;
size_t belowThreshold = 0;
};
/**
* @brief Return the sum of IBF decode failure counters among created nodes.
*/
IbfDecodeFailureCounts
countIbfDecodeFailures() const
{
IbfDecodeFailureCounts result;
for (const auto& node : nodes) {
if (node == nullptr) {
continue;
}
result.aboveThreshold += node->nIbfDecodeFailuresAboveThreshold;
result.belowThreshold += node->nIbfDecodeFailuresBelowThreshold;
}
return result;
}
/**
* @brief Repeat a test function until there are IBF decode failures.
* @param minTotalUpdates minimum totalUpdates parameter.
* @param maxTotalUpdates maximum totalUpdates parameter.
* @param f test function.
*
* This method searches for totalUpdates [minTotalUpdates,maxTotalUpdates] until
* there is at least one execution that caused an IBF decode failure above threshold.
* If such an execution never occurs within the range, the test case fails.
*
* Current FullSync logic cannot reliably recover from an IBF decode failure below threshold.
* Hence, that condition is not tested.
*/
void
searchIbfDecodeFailures(int minTotalUpdates, int maxTotalUpdates,
const std::function<void(int totalUpdates)>& f)
{
bool hasAboveThreshold = false;
for (int totalUpdates = minTotalUpdates; totalUpdates <= maxTotalUpdates; ++totalUpdates) {
clearNodes();
BOOST_TEST_CONTEXT("totalUpdates=" << totalUpdates) {
f(totalUpdates);
auto cnt = countIbfDecodeFailures();
BOOST_TEST_MESSAGE("aboveThreshold=" << cnt.aboveThreshold << " "
"belowThreshold=" << cnt.belowThreshold);
hasAboveThreshold = hasAboveThreshold || cnt.aboveThreshold > 0;
if (hasAboveThreshold) {
return;
}
}
}
BOOST_TEST_FAIL("cannot find viable totalUpdates for IBF decode failures");
}
protected:
const Name syncPrefix = "/psync";
shared_ptr<util::DummyClientFace> faces[4];
Name userPrefixes[4];
shared_ptr<FullProducer> nodes[4];
static constexpr int MAX_NODES = 4;
std::array<Name, MAX_NODES> userPrefixes;
std::array<std::unique_ptr<ndn::DummyClientFace>, MAX_NODES> faces;
std::array<std::unique_ptr<FullProducer>, MAX_NODES> nodes;
static constexpr uint64_t NOT_EXIST = std::numeric_limits<uint64_t>::max();
};
BOOST_FIXTURE_TEST_SUITE(TestFullSync, FullSyncFixture)
@@ -60,23 +188,23 @@ BOOST_AUTO_TEST_CASE(TwoNodesSimple)
addNode(1);
faces[0]->linkTo(*faces[1]);
advanceClocks(ndn::time::milliseconds(10));
advanceClocks(10_ms);
nodes[0]->publishName(userPrefixes[0]);
advanceClocks(ndn::time::milliseconds(10), 100);
advanceClocks(10_ms, 100);
BOOST_CHECK_EQUAL(nodes[0]->getSeqNo(userPrefixes[0]).value_or(-1), 1);
BOOST_CHECK_EQUAL(nodes[1]->getSeqNo(userPrefixes[0]).value_or(-1), 1);
BOOST_CHECK_EQUAL(nodes[0]->getSeqNo(userPrefixes[0]).value_or(NOT_EXIST), 1);
BOOST_CHECK_EQUAL(nodes[1]->getSeqNo(userPrefixes[0]).value_or(NOT_EXIST), 1);
nodes[1]->publishName(userPrefixes[1]);
advanceClocks(ndn::time::milliseconds(10), 100);
BOOST_CHECK_EQUAL(nodes[0]->getSeqNo(userPrefixes[1]).value_or(-1), 1);
BOOST_CHECK_EQUAL(nodes[1]->getSeqNo(userPrefixes[1]).value_or(-1), 1);
advanceClocks(10_ms, 100);
BOOST_CHECK_EQUAL(nodes[0]->getSeqNo(userPrefixes[1]).value_or(NOT_EXIST), 1);
BOOST_CHECK_EQUAL(nodes[1]->getSeqNo(userPrefixes[1]).value_or(NOT_EXIST), 1);
nodes[1]->publishName(userPrefixes[1]);
advanceClocks(ndn::time::milliseconds(10), 100);
BOOST_CHECK_EQUAL(nodes[0]->getSeqNo(userPrefixes[1]).value_or(-1), 2);
BOOST_CHECK_EQUAL(nodes[1]->getSeqNo(userPrefixes[1]).value_or(-1), 2);
advanceClocks(10_ms, 100);
BOOST_CHECK_EQUAL(nodes[0]->getSeqNo(userPrefixes[1]).value_or(NOT_EXIST), 2);
BOOST_CHECK_EQUAL(nodes[1]->getSeqNo(userPrefixes[1]).value_or(NOT_EXIST), 2);
}
BOOST_AUTO_TEST_CASE(TwoNodesForceSeqNo)
@@ -85,13 +213,13 @@ BOOST_AUTO_TEST_CASE(TwoNodesForceSeqNo)
addNode(1);
faces[0]->linkTo(*faces[1]);
advanceClocks(ndn::time::milliseconds(10));
advanceClocks(10_ms);
nodes[0]->publishName(userPrefixes[0], 3);
advanceClocks(ndn::time::milliseconds(10), 100);
advanceClocks(10_ms, 100);
BOOST_CHECK_EQUAL(nodes[0]->getSeqNo(userPrefixes[0]).value_or(-1), 3);
BOOST_CHECK_EQUAL(nodes[1]->getSeqNo(userPrefixes[0]).value_or(-1), 3);
BOOST_CHECK_EQUAL(nodes[0]->getSeqNo(userPrefixes[0]).value_or(NOT_EXIST), 3);
BOOST_CHECK_EQUAL(nodes[1]->getSeqNo(userPrefixes[0]).value_or(NOT_EXIST), 3);
}
BOOST_AUTO_TEST_CASE(TwoNodesWithMultipleUserNodes)
@@ -100,31 +228,31 @@ BOOST_AUTO_TEST_CASE(TwoNodesWithMultipleUserNodes)
addNode(1);
faces[0]->linkTo(*faces[1]);
advanceClocks(ndn::time::milliseconds(10));
advanceClocks(10_ms);
Name nodeZeroExtraUser("userPrefix0-1");
Name nodeOneExtraUser("userPrefix1-1");
Name nodeZeroExtraUser("/userPrefix0-1");
Name nodeOneExtraUser("/userPrefix1-1");
nodes[0]->addUserNode(nodeZeroExtraUser);
nodes[1]->addUserNode(nodeOneExtraUser);
nodes[0]->publishName(userPrefixes[0]);
advanceClocks(ndn::time::milliseconds(10), 100);
advanceClocks(10_ms, 100);
BOOST_CHECK_EQUAL(nodes[0]->getSeqNo(userPrefixes[0]).value_or(-1), 1);
BOOST_CHECK_EQUAL(nodes[1]->getSeqNo(userPrefixes[0]).value_or(-1), 1);
BOOST_CHECK_EQUAL(nodes[0]->getSeqNo(userPrefixes[0]).value_or(NOT_EXIST), 1);
BOOST_CHECK_EQUAL(nodes[1]->getSeqNo(userPrefixes[0]).value_or(NOT_EXIST), 1);
nodes[0]->publishName(nodeZeroExtraUser);
advanceClocks(ndn::time::milliseconds(10), 100);
advanceClocks(10_ms, 100);
BOOST_CHECK_EQUAL(nodes[0]->getSeqNo(nodeZeroExtraUser).value_or(-1), 1);
BOOST_CHECK_EQUAL(nodes[1]->getSeqNo(nodeZeroExtraUser).value_or(-1), 1);
BOOST_CHECK_EQUAL(nodes[0]->getSeqNo(nodeZeroExtraUser).value_or(NOT_EXIST), 1);
BOOST_CHECK_EQUAL(nodes[1]->getSeqNo(nodeZeroExtraUser).value_or(NOT_EXIST), 1);
nodes[1]->publishName(nodeOneExtraUser);
advanceClocks(ndn::time::milliseconds(10), 100);
advanceClocks(10_ms, 100);
BOOST_CHECK_EQUAL(nodes[0]->getSeqNo(nodeOneExtraUser).value_or(-1), 1);
BOOST_CHECK_EQUAL(nodes[1]->getSeqNo(nodeOneExtraUser).value_or(-1), 1);
BOOST_CHECK_EQUAL(nodes[0]->getSeqNo(nodeOneExtraUser).value_or(NOT_EXIST), 1);
BOOST_CHECK_EQUAL(nodes[1]->getSeqNo(nodeOneExtraUser).value_or(NOT_EXIST), 1);
}
BOOST_AUTO_TEST_CASE(MultipleNodes)
@@ -137,21 +265,21 @@ BOOST_AUTO_TEST_CASE(MultipleNodes)
}
nodes[0]->publishName(userPrefixes[0]);
advanceClocks(ndn::time::milliseconds(10), 100);
advanceClocks(10_ms, 100);
for (int i = 0; i < 4; i++) {
BOOST_CHECK_EQUAL(nodes[i]->getSeqNo(userPrefixes[0]).value_or(-1), 1);
BOOST_CHECK_EQUAL(nodes[i]->getSeqNo(userPrefixes[0]).value_or(NOT_EXIST), 1);
}
nodes[1]->publishName(userPrefixes[1]);
advanceClocks(ndn::time::milliseconds(10), 100);
advanceClocks(10_ms, 100);
for (int i = 0; i < 4; i++) {
BOOST_CHECK_EQUAL(nodes[i]->getSeqNo(userPrefixes[1]).value_or(-1), 1);
BOOST_CHECK_EQUAL(nodes[i]->getSeqNo(userPrefixes[1]).value_or(NOT_EXIST), 1);
}
nodes[1]->publishName(userPrefixes[1]);
advanceClocks(ndn::time::milliseconds(10), 100);
advanceClocks(10_ms, 100);
for (int i = 0; i < 4; i++) {
BOOST_CHECK_EQUAL(nodes[i]->getSeqNo(userPrefixes[1]).value_or(-1), 2);
BOOST_CHECK_EQUAL(nodes[i]->getSeqNo(userPrefixes[1]).value_or(NOT_EXIST), 2);
}
}
@@ -167,10 +295,10 @@ BOOST_AUTO_TEST_CASE(MultipleNodesSimultaneousPublish)
nodes[i]->publishName(userPrefixes[i]);
}
advanceClocks(ndn::time::milliseconds(10), 100);
advanceClocks(10_ms, 100);
for (int i = 0; i < 4; i++) {
for (int j = 0; j < 4; j++) {
BOOST_CHECK_EQUAL(nodes[i]->getSeqNo(userPrefixes[j]).value_or(-1), 1);
BOOST_CHECK_EQUAL(nodes[i]->getSeqNo(userPrefixes[j]).value_or(NOT_EXIST), 1);
}
}
@@ -178,10 +306,10 @@ BOOST_AUTO_TEST_CASE(MultipleNodesSimultaneousPublish)
nodes[i]->publishName(userPrefixes[i], 4);
}
advanceClocks(ndn::time::milliseconds(10), 100);
advanceClocks(10_ms, 100);
for (int i = 0; i < 4; i++) {
for (int j = 0; j < 4; j++) {
BOOST_CHECK_EQUAL(nodes[i]->getSeqNo(userPrefixes[j]).value_or(-1), 4);
BOOST_CHECK_EQUAL(nodes[i]->getSeqNo(userPrefixes[j]).value_or(NOT_EXIST), 4);
}
}
}
@@ -196,9 +324,9 @@ BOOST_AUTO_TEST_CASE(NetworkPartition)
}
nodes[0]->publishName(userPrefixes[0]);
advanceClocks(ndn::time::milliseconds(10), 100);
advanceClocks(10_ms, 100);
for (int i = 0; i < 4; i++) {
BOOST_CHECK_EQUAL(nodes[i]->getSeqNo(userPrefixes[0]).value_or(-1), 1);
BOOST_CHECK_EQUAL(nodes[i]->getSeqNo(userPrefixes[0]).value_or(NOT_EXIST), 1);
}
for (int i = 0; i < 3; i++) {
@@ -208,25 +336,25 @@ BOOST_AUTO_TEST_CASE(NetworkPartition)
faces[2]->linkTo(*faces[3]);
nodes[0]->publishName(userPrefixes[0]);
advanceClocks(ndn::time::milliseconds(10), 100);
BOOST_CHECK_EQUAL(nodes[1]->getSeqNo(userPrefixes[0]).value_or(-1), 2);
BOOST_CHECK_EQUAL(nodes[2]->getSeqNo(userPrefixes[0]).value_or(-1), 1);
BOOST_CHECK_EQUAL(nodes[3]->getSeqNo(userPrefixes[0]).value_or(-1), 1);
advanceClocks(10_ms, 100);
BOOST_CHECK_EQUAL(nodes[1]->getSeqNo(userPrefixes[0]).value_or(NOT_EXIST), 2);
BOOST_CHECK_EQUAL(nodes[2]->getSeqNo(userPrefixes[0]).value_or(NOT_EXIST), 1);
BOOST_CHECK_EQUAL(nodes[3]->getSeqNo(userPrefixes[0]).value_or(NOT_EXIST), 1);
nodes[1]->publishName(userPrefixes[1], 2);
advanceClocks(ndn::time::milliseconds(10), 100);
BOOST_CHECK_EQUAL(nodes[0]->getSeqNo(userPrefixes[1]).value_or(-1), 2);
advanceClocks(10_ms, 100);
BOOST_CHECK_EQUAL(nodes[0]->getSeqNo(userPrefixes[1]).value_or(NOT_EXIST), 2);
nodes[2]->publishName(userPrefixes[2], 2);
advanceClocks(ndn::time::milliseconds(10), 100);
BOOST_CHECK_EQUAL(nodes[3]->getSeqNo(userPrefixes[2]).value_or(-1), 2);
advanceClocks(10_ms, 100);
BOOST_CHECK_EQUAL(nodes[3]->getSeqNo(userPrefixes[2]).value_or(NOT_EXIST), 2);
nodes[3]->publishName(userPrefixes[3], 2);
advanceClocks(ndn::time::milliseconds(10), 100);
BOOST_CHECK_EQUAL(nodes[2]->getSeqNo(userPrefixes[3]).value_or(-1), 2);
advanceClocks(10_ms, 100);
BOOST_CHECK_EQUAL(nodes[2]->getSeqNo(userPrefixes[3]).value_or(NOT_EXIST), 2);
BOOST_CHECK_EQUAL(nodes[0]->getSeqNo(userPrefixes[3]).value_or(-1), -1);
BOOST_CHECK_EQUAL(nodes[1]->getSeqNo(userPrefixes[3]).value_or(-1), -1);
BOOST_CHECK_EQUAL(nodes[0]->getSeqNo(userPrefixes[3]).value_or(NOT_EXIST), NOT_EXIST);
BOOST_CHECK_EQUAL(nodes[1]->getSeqNo(userPrefixes[3]).value_or(NOT_EXIST), NOT_EXIST);
for (int i = 0; i < 3; i++) {
faces[i]->unlink();
@@ -235,10 +363,10 @@ BOOST_AUTO_TEST_CASE(NetworkPartition)
faces[i]->linkTo(*faces[i + 1]);
}
advanceClocks(ndn::time::milliseconds(10), 100);
advanceClocks(10_ms, 100);
for (int i = 0; i < 4; i++) {
for (int j = 0; j < 4; j++) {
BOOST_CHECK_EQUAL(nodes[i]->getSeqNo(userPrefixes[j]).value_or(-1), 2);
BOOST_CHECK_EQUAL(nodes[i]->getSeqNo(userPrefixes[j]).value_or(NOT_EXIST), 2);
}
}
}
@@ -249,140 +377,89 @@ BOOST_AUTO_TEST_CASE(IBFOverflow)
addNode(1);
faces[0]->linkTo(*faces[1]);
advanceClocks(ndn::time::milliseconds(10));
advanceClocks(10_ms);
// 50 > 40 (expected number of entries in IBF)
for (int i = 0; i < 50; i++) {
nodes[0]->addUserNode(Name("userNode0-" + to_string(i)));
nodes[0]->addUserNode(makeSubPrefix(0, i));
}
batchUpdate(0, 0, 20, 1);
advanceClocks(10_ms, 100);
batchCheck(1, 0, 0, 20, 1);
for (int i = 0; i < 20; i++) {
// Suppose all sync data were lost for these:
nodes[0]->updateSeqNo(Name("userNode0-" + to_string(i)), 1);
}
nodes[0]->publishName(Name("userNode0-" + to_string(20)));
advanceClocks(ndn::time::milliseconds(10), 100);
for (int i = 0; i <= 20; i++) {
Name userPrefix("userNode0-" + to_string(i));
BOOST_CHECK_EQUAL(nodes[1]->getSeqNo(userPrefix).value_or(-1), 1);
}
for (int i = 21; i < 49; i++) {
nodes[0]->updateSeqNo(Name("userNode0-" + to_string(i)), 1);
}
nodes[0]->publishName(Name("userNode0-49"));
advanceClocks(ndn::time::milliseconds(10), 100);
for (int i = 21; i < 49; i++) {
Name userPrefix("userNode0-" + to_string(i));
BOOST_CHECK_EQUAL(nodes[1]->getSeqNo(userPrefix).value_or(-1), 1);
}
batchUpdate(0, 21, 49, 1);
advanceClocks(10_ms, 100);
batchCheck(1, 0, 21, 49, 1);
}
BOOST_AUTO_TEST_CASE(DiffIBFDecodeFailureSimple)
{
addNode(0);
addNode(1);
searchIbfDecodeFailures(46, 52, [this] (int totalUpdates) {
addNode(0);
addNode(1);
faces[0]->linkTo(*faces[1]);
advanceClocks(ndn::time::milliseconds(10));
faces[0]->linkTo(*faces[1]);
advanceClocks(10_ms);
// Lowest number that triggers a decode failure for IBF size of 40
int totalUpdates = 47;
batchUpdate(0, 0, totalUpdates, 1);
advanceClocks(10_ms, 100);
batchCheck(1, 0, 0, totalUpdates, 1);
for (int i = 0; i <= totalUpdates; i++) {
nodes[0]->addUserNode(Name("userNode0-" + to_string(i)));
if (i != totalUpdates) {
nodes[0]->updateSeqNo(Name("userNode0-" + to_string(i)), 1);
}
}
nodes[0]->publishName(Name("userNode0-" + to_string(totalUpdates)));
advanceClocks(ndn::time::milliseconds(10), 100);
BOOST_CHECK_EQUAL(nodes[0]->getSeqNo(userPrefixes[1]).value_or(NOT_EXIST), NOT_EXIST);
BOOST_CHECK_EQUAL(nodes[1]->getSeqNo(userPrefixes[0]).value_or(NOT_EXIST), NOT_EXIST);
// No mechanism to recover yet
for (int i = 0; i <= totalUpdates; i++) {
Name userPrefix("userNode0-" + to_string(i));
BOOST_CHECK_EQUAL(nodes[1]->getSeqNo(userPrefix).value_or(-1), 1);
}
nodes[1]->publishName(userPrefixes[1]);
advanceClocks(10_ms, 100);
BOOST_CHECK_EQUAL(nodes[0]->getSeqNo(userPrefixes[1]).value_or(NOT_EXIST), 1);
BOOST_CHECK_EQUAL(nodes[0]->getSeqNo(userPrefixes[1]).value_or(-1), -1);
BOOST_CHECK_EQUAL(nodes[1]->getSeqNo(userPrefixes[0]).value_or(-1), -1);
nodes[1]->publishName(userPrefixes[1]);
advanceClocks(ndn::time::milliseconds(10), 100);
BOOST_CHECK_EQUAL(nodes[0]->getSeqNo(userPrefixes[1]).value_or(-1), 1);
nodes[0]->publishName(userPrefixes[0]);
advanceClocks(ndn::time::milliseconds(10), 100);
BOOST_CHECK_EQUAL(nodes[1]->getSeqNo(userPrefixes[0]).value_or(-1), 1);
nodes[0]->publishName(userPrefixes[0]);
advanceClocks(10_ms, 100);
BOOST_CHECK_EQUAL(nodes[1]->getSeqNo(userPrefixes[0]).value_or(NOT_EXIST), 1);
});
}
BOOST_AUTO_TEST_CASE(DiffIBFDecodeFailureSimpleSegmentedRecovery)
{
addNode(0);
addNode(1);
faces[0]->linkTo(*faces[1]);
searchIbfDecodeFailures(46, 52, [this] (int totalUpdates) {
addNode(0);
addNode(1);
faces[0]->linkTo(*faces[1]);
advanceClocks(ndn::time::milliseconds(10));
advanceClocks(10_ms);
// Lowest number that triggers a decode failure for IBF size of 40
int totalUpdates = 270;
batchUpdate(0, 0, totalUpdates, 1);
advanceClocks(10_ms, 100);
batchCheck(1, 0, 0, totalUpdates, 1);
for (int i = 0; i <= totalUpdates; i++) {
nodes[0]->addUserNode(Name("userNode0-" + to_string(i)));
if (i != totalUpdates) {
nodes[0]->updateSeqNo(Name("userNode0-" + to_string(i)), 1);
}
}
nodes[0]->publishName(Name("userNode0-" + to_string(totalUpdates)));
advanceClocks(ndn::time::milliseconds(10), 100);
BOOST_CHECK_EQUAL(nodes[0]->getSeqNo(userPrefixes[1]).value_or(NOT_EXIST), NOT_EXIST);
BOOST_CHECK_EQUAL(nodes[1]->getSeqNo(userPrefixes[0]).value_or(NOT_EXIST), NOT_EXIST);
// No mechanism to recover yet
for (int i = 0; i <= totalUpdates; i++) {
Name userPrefix("userNode0-" + to_string(i));
BOOST_CHECK_EQUAL(nodes[1]->getSeqNo(userPrefix).value_or(-1), 1);
}
nodes[1]->publishName(userPrefixes[1]);
advanceClocks(10_ms, 100);
BOOST_CHECK_EQUAL(nodes[0]->getSeqNo(userPrefixes[1]).value_or(NOT_EXIST), 1);
BOOST_CHECK_EQUAL(nodes[0]->getSeqNo(userPrefixes[1]).value_or(-1), -1);
BOOST_CHECK_EQUAL(nodes[1]->getSeqNo(userPrefixes[0]).value_or(-1), -1);
nodes[1]->publishName(userPrefixes[1]);
advanceClocks(ndn::time::milliseconds(10), 100);
BOOST_CHECK_EQUAL(nodes[0]->getSeqNo(userPrefixes[1]).value_or(-1), 1);
nodes[0]->publishName(userPrefixes[0]);
advanceClocks(ndn::time::milliseconds(10), 100);
BOOST_CHECK_EQUAL(nodes[1]->getSeqNo(userPrefixes[0]).value_or(-1), 1);
nodes[0]->publishName(userPrefixes[0]);
advanceClocks(10_ms, 100);
BOOST_CHECK_EQUAL(nodes[1]->getSeqNo(userPrefixes[0]).value_or(NOT_EXIST), 1);
});
}
BOOST_AUTO_TEST_CASE(DiffIBFDecodeFailureMultipleNodes)
{
for (int i = 0; i < 4; i++) {
addNode(i);
}
for (int i = 0; i < 3; i++) {
faces[i]->linkTo(*faces[i + 1]);
}
// Lowest number that triggers a decode failure for IBF size of 40
int totalUpdates = 47;
for (int i = 0; i <= totalUpdates; i++) {
nodes[0]->addUserNode(Name("userNode0-" + to_string(i)));
if (i != totalUpdates) {
nodes[0]->updateSeqNo(Name("userNode0-" + to_string(i)), 1);
searchIbfDecodeFailures(46, 52, [this] (int totalUpdates) {
for (int i = 0; i < 4; i++) {
addNode(i);
}
}
nodes[0]->publishName(Name("userNode0-" + to_string(totalUpdates)));
advanceClocks(ndn::time::milliseconds(10), 100);
for (int i = 0; i <= totalUpdates; i++) {
Name userPrefix("userNode0-" + to_string(i));
for (int j = 0; j < 4; j++) {
BOOST_CHECK_EQUAL(nodes[j]->getSeqNo(userPrefix).value_or(-1), 1);
for (int i = 0; i < 3; i++) {
faces[i]->linkTo(*faces[i + 1]);
}
}
batchUpdate(0, 0, totalUpdates, 1);
advanceClocks(10_ms, 100);
for (int i = 0; i < 4; i++) {
batchCheck(i, 0, 0, totalUpdates, 1);
}
});
}
BOOST_AUTO_TEST_CASE(DelayedSecondSegment)
@@ -391,19 +468,19 @@ BOOST_AUTO_TEST_CASE(DelayedSecondSegment)
int i = 0;
detail::State state;
std::shared_ptr<Buffer> compressed;
std::shared_ptr<ndn::Buffer> compressed;
do {
Name prefixToPublish("userNode0-" + to_string(i++));
auto prefixToPublish = makeSubPrefix(0, i++);
nodes[0]->addUserNode(prefixToPublish);
nodes[0]->publishName(prefixToPublish);
state.addContent(Name(prefixToPublish).appendNumber(nodes[0]->m_prefixes[prefixToPublish]));
auto block = state.wireEncode();
compressed = detail::compress(nodes[0]->m_contentCompression, block.wire(), block.size());
} while (compressed->size() < (MAX_NDN_PACKET_SIZE >> 1));
compressed = detail::compress(nodes[0]->m_contentCompression, block);
} while (compressed->size() < (ndn::MAX_NDN_PACKET_SIZE >> 1));
advanceClocks(ndn::time::milliseconds(10), 100);
advanceClocks(10_ms, 100);
Name syncInterestName(syncPrefix);
detail::IBLT iblt(40, nodes[0]->m_ibltCompression);
@@ -411,11 +488,11 @@ BOOST_AUTO_TEST_CASE(DelayedSecondSegment)
nodes[0]->onSyncInterest(syncPrefix, Interest(syncInterestName));
advanceClocks(ndn::time::milliseconds(10));
advanceClocks(10_ms);
BOOST_CHECK_EQUAL(nodes[0]->m_segmentPublisher.m_ims.size(), 2);
// Expire contents from segmentPublisher
advanceClocks(ndn::time::milliseconds(10), 100);
advanceClocks(10_ms, 100);
BOOST_CHECK_EQUAL(nodes[0]->m_segmentPublisher.m_ims.size(), 0);
// Get data name from face and increase segment number to form next interest
@@ -426,7 +503,7 @@ BOOST_AUTO_TEST_CASE(DelayedSecondSegment)
faces[0]->sentData.clear();
nodes[0]->onSyncInterest(syncPrefix, Interest(interestName));
advanceClocks(ndn::time::milliseconds(10));
advanceClocks(10_ms);
// Should have repopulated SegmentPublisher
BOOST_CHECK_EQUAL(nodes[0]->m_segmentPublisher.m_ims.size(), 2);
+24 -27
View File
@@ -1,6 +1,6 @@
/* -*- Mode:C++; c-file-style:"gnu"; indent-tabs-mode:nil; -*- */
/*
* Copyright (c) 2014-2020, The University of Memphis
* Copyright (c) 2014-2023, The University of Memphis
*
* This file is part of PSync.
* See AUTHORS.md for complete list of PSync authors and contributors.
@@ -25,7 +25,7 @@
namespace psync {
namespace detail {
using namespace ndn;
using ndn::Name;
BOOST_AUTO_TEST_SUITE(TestIBLT)
@@ -37,7 +37,7 @@ BOOST_AUTO_TEST_CASE(Equal)
IBLT iblt2(size, CompressionScheme::DEFAULT);
BOOST_CHECK_EQUAL(iblt1, iblt2);
std::string prefix = Name("/test/memphis").appendNumber(1).toUri();
auto prefix = Name("/test/memphis").appendNumber(1);
uint32_t newHash = murmurHash3(11, prefix);
iblt1.insert(newHash);
iblt2.insert(newHash);
@@ -54,17 +54,17 @@ static const uint8_t IBF[] = {
// Header
0x08, 0xB4,
// Uncompressed IBF
0x00, 0x00, 0x00, 0x01, 0xF6, 0xA7, 0x7A, 0xBA, 0x6B, 0xA3, 0x4D, 0x63, 0x00, 0x00, 0x00, 0x00,
0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x01, 0x5C, 0x5B, 0xF2, 0x67,
0x42, 0x24, 0xEE, 0x6C, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x01, 0x5C, 0x5B, 0xF2, 0x67,
0x42, 0x24, 0xEE, 0x6C, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x01, 0xF6, 0xA7, 0x7A, 0xBA,
0x6B, 0xA3, 0x4D, 0x63, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
0x00, 0x00, 0x00, 0x01, 0xF6, 0xA7, 0x7A, 0xBA, 0x6B, 0xA3, 0x4D, 0x63, 0x00, 0x00, 0x00, 0x00,
0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x01,
0x5C, 0x5B, 0xF2, 0x67, 0x42, 0x24, 0xEE, 0x6C, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
0x00, 0x00, 0x00, 0x00
};
@@ -73,7 +73,7 @@ BOOST_AUTO_TEST_CASE(NameAppendAndExtract)
const int size = 10;
IBLT iblt(size, CompressionScheme::DEFAULT);
auto prefix = Name("/test/memphis").appendNumber(1).toUri();
auto prefix = Name("/test/memphis").appendNumber(1);
auto hash = murmurHash3(11, prefix);
iblt.insert(hash);
@@ -91,15 +91,12 @@ BOOST_AUTO_TEST_CASE(NameAppendAndExtract)
iblt2.insert(hash);
Name ibltName2("/sync");
iblt2.appendToName(ibltName2);
BOOST_CHECK_EQUAL_COLLECTIONS(IBF, IBF + sizeof(IBF),
ibltName2.at(-1).wireEncode().begin(),
ibltName2.at(-1).wireEncode().end());
BOOST_TEST(ibltName2.at(-1).wireEncode() == IBF, boost::test_tools::per_element());
Name malformedName("/test");
auto compressed = compress(CompressionScheme::DEFAULT,
malformedName.wireEncode().value(),
malformedName.wireEncode().value_size());
malformedName.append(compressed->data(), compressed->size());
malformedName.wireEncode().value_bytes());
malformedName.append(Name::Component(std::move(compressed)));
IBLT rcvd2(size, CompressionScheme::DEFAULT);
BOOST_CHECK_THROW(rcvd2.initialize(malformedName.at(-1)), IBLT::Error);
}
@@ -110,18 +107,18 @@ BOOST_AUTO_TEST_CASE(CopyInsertErase)
IBLT iblt1(size, CompressionScheme::DEFAULT);
std::string prefix = Name("/test/memphis").appendNumber(1).toUri();
auto prefix = Name("/test/memphis").appendNumber(1);
uint32_t hash1 = murmurHash3(11, prefix);
iblt1.insert(hash1);
IBLT iblt2(iblt1);
iblt2.erase(hash1);
prefix = Name("/test/memphis").appendNumber(2).toUri();
prefix = Name("/test/memphis").appendNumber(2);
uint32_t hash3 = murmurHash3(11, prefix);
iblt2.insert(hash3);
iblt1.erase(hash1);
prefix = Name("/test/memphis").appendNumber(5).toUri();
prefix = Name("/test/memphis").appendNumber(5);
uint32_t hash5 = murmurHash3(11, prefix);
iblt1.insert(hash5);
@@ -141,11 +138,11 @@ BOOST_AUTO_TEST_CASE(HigherSeqTest)
IBLT ownIBF(size, CompressionScheme::DEFAULT);
IBLT rcvdIBF(size, CompressionScheme::DEFAULT);
std::string prefix = Name("/test/memphis").appendNumber(3).toUri();
auto prefix = Name("/test/memphis").appendNumber(3);
uint32_t hash1 = murmurHash3(11, prefix);
ownIBF.insert(hash1);
std::string prefix2 = Name("/test/memphis").appendNumber(4).toUri();
auto prefix2 = Name("/test/memphis").appendNumber(4);
uint32_t hash2 = murmurHash3(11, prefix2);
rcvdIBF.insert(hash2);
@@ -174,7 +171,7 @@ BOOST_AUTO_TEST_CASE(Difference)
BOOST_CHECK_EQUAL(positive.size(), 0);
BOOST_CHECK_EQUAL(negative.size(), 0);
std::string prefix = Name("/test/memphis").appendNumber(1).toUri();
auto prefix = Name("/test/memphis").appendNumber(1);
uint32_t newHash = murmurHash3(11, prefix);
ownIBF.insert(newHash);
@@ -183,7 +180,7 @@ BOOST_AUTO_TEST_CASE(Difference)
BOOST_CHECK_EQUAL(positive.size(), 1);
BOOST_CHECK_EQUAL(negative.size(), 0);
prefix = Name("/test/csu").appendNumber(1).toUri();
prefix = Name("/test/csu").appendNumber(1);
newHash = murmurHash3(11, prefix);
rcvdIBF.insert(newHash);
@@ -204,14 +201,14 @@ BOOST_AUTO_TEST_CASE(DifferenceBwOversizedIBFs)
IBLT ownIBF(size, CompressionScheme::DEFAULT);
for (int i = 0; i < 50; i++) {
std::string prefix = Name("/test/memphis" + std::to_string(i)).appendNumber(1).toUri();
auto prefix = Name("/test/memphis" + std::to_string(i)).appendNumber(1);
uint32_t newHash = murmurHash3(11, prefix);
ownIBF.insert(newHash);
}
IBLT rcvdIBF = ownIBF;
std::string prefix = Name("/test/ucla").appendNumber(1).toUri();
auto prefix = Name("/test/ucla").appendNumber(1);
uint32_t newHash = murmurHash3(11, prefix);
ownIBF.insert(newHash);
+24 -27
View File
@@ -1,6 +1,6 @@
/* -*- Mode:C++; c-file-style:"gnu"; indent-tabs-mode:nil; -*- */
/*
* Copyright (c) 2014-2020, The University of Memphis
* Copyright (c) 2014-2023, The University of Memphis
*
* This file is part of PSync.
* See AUTHORS.md for complete list of PSync authors and contributors.
@@ -20,43 +20,42 @@
#include "PSync/partial-producer.hpp"
#include "tests/boost-test.hpp"
#include "tests/key-chain-fixture.hpp"
#include <ndn-cxx/name.hpp>
#include <ndn-cxx/mgmt/nfd/control-parameters.hpp>
#include <ndn-cxx/util/dummy-client-face.hpp>
namespace psync {
using namespace ndn;
using ndn::Interest;
using ndn::Name;
BOOST_AUTO_TEST_SUITE(TestPartialProducer)
BOOST_AUTO_TEST_CASE(Constructor)
class PartialProducerFixture : public tests::KeyChainFixture
{
util::DummyClientFace face({true, true});
BOOST_CHECK_NO_THROW(PartialProducer(40, face, Name("/psync"), Name("/testUser")));
}
protected:
ndn::DummyClientFace m_face{m_keyChain, {true, true}};
};
BOOST_FIXTURE_TEST_SUITE(TestPartialProducer, PartialProducerFixture)
BOOST_AUTO_TEST_CASE(RegisterPrefix)
{
Name syncPrefix("/psync"), userNode("/testUser");
util::DummyClientFace face({true, true});
PartialProducer producer(40, face, syncPrefix, userNode);
PartialProducer producer(m_face, m_keyChain, 40, syncPrefix, userNode);
face.processEvents(-1_ms);
m_face.processEvents(-1_ms);
BOOST_REQUIRE_EQUAL(face.sentInterests.size(), 1);
auto interest = face.sentInterests.front();
BOOST_CHECK_EQUAL(interest.getName().at(3), name::Component("register"));
nfd::ControlParameters params(interest.getName().at(4).blockFromValue());
BOOST_REQUIRE_EQUAL(m_face.sentInterests.size(), 1);
auto interest = m_face.sentInterests.front();
BOOST_CHECK_EQUAL(interest.getName().at(3), Name::Component("register"));
ndn::nfd::ControlParameters params(interest.getName().at(4).blockFromValue());
BOOST_CHECK_EQUAL(params.getName(), syncPrefix);
}
BOOST_AUTO_TEST_CASE(PublishName)
{
Name syncPrefix("/psync"), userNode("/testUser"), nonUser("/testUser2");
util::DummyClientFace face({true, true});
PartialProducer producer(40, face, syncPrefix, userNode);
PartialProducer producer(m_face, m_keyChain, 40, syncPrefix, userNode);
BOOST_CHECK_EQUAL(producer.getSeqNo(userNode).value_or(-1), 0);
producer.publishName(userNode);
@@ -75,8 +74,7 @@ BOOST_AUTO_TEST_CASE(PublishName)
BOOST_AUTO_TEST_CASE(SameSyncInterest)
{
Name syncPrefix("/psync"), userNode("/testUser");
util::DummyClientFace face({true, true});
PartialProducer producer(40, face, syncPrefix, userNode);
PartialProducer producer(m_face, m_keyChain, 40, syncPrefix, userNode);
Name syncInterestName(syncPrefix);
syncInterestName.append("sync");
@@ -91,29 +89,28 @@ BOOST_AUTO_TEST_CASE(SameSyncInterest)
syncInterest.setInterestLifetime(1_s);
syncInterest.setNonce(1);
BOOST_CHECK_NO_THROW(producer.onSyncInterest(syncInterestPrefix, syncInterest));
face.processEvents(10_ms);
m_face.processEvents(10_ms);
BOOST_CHECK_EQUAL(producer.m_pendingEntries.size(), 1);
face.processEvents(500_ms);
m_face.processEvents(500_ms);
// Same interest again - size of pending interest should remain same, but expirationEvent should change
syncInterest.setNonce(2);
BOOST_CHECK_NO_THROW(producer.onSyncInterest(syncInterestPrefix, syncInterest));
face.processEvents(10_ms);
m_face.processEvents(10_ms);
BOOST_CHECK_EQUAL(producer.m_pendingEntries.size(), 1);
face.processEvents(500_ms);
m_face.processEvents(500_ms);
BOOST_CHECK_EQUAL(producer.m_pendingEntries.size(), 1);
face.processEvents(500_ms);
m_face.processEvents(500_ms);
BOOST_CHECK_EQUAL(producer.m_pendingEntries.size(), 0);
}
BOOST_AUTO_TEST_CASE(OnSyncInterest)
{
Name syncPrefix("/psync"), userNode("/testUser");
util::DummyClientFace face({true, true});
PartialProducer producer(40, face, syncPrefix, userNode);
PartialProducer producer(m_face, m_keyChain, 40, syncPrefix, userNode);
// Sync interest with no bloom filter attached
Name syncInterestName(syncPrefix);
+26 -24
View File
@@ -1,6 +1,6 @@
/* -*- Mode:C++; c-file-style:"gnu"; indent-tabs-mode:nil; -*- */
/*
* Copyright (c) 2014-2020, The University of Memphis
* Copyright (c) 2014-2023, The University of Memphis
*
* This file is part of PSync.
* See AUTHORS.md for complete list of PSync authors and contributors.
@@ -22,20 +22,22 @@
#include "tests/boost-test.hpp"
#include "tests/io-fixture.hpp"
#include "tests/key-chain-fixture.hpp"
#include <ndn-cxx/name.hpp>
#include <array>
#include <ndn-cxx/util/dummy-client-face.hpp>
namespace psync {
using namespace ndn;
using ndn::Interest;
using ndn::Name;
class PartialSyncFixture : public tests::IoFixture
class PartialSyncFixture : public tests::IoFixture, public tests::KeyChainFixture
{
public:
protected:
PartialSyncFixture()
{
producer = make_shared<PartialProducer>(40, face, syncPrefix, userPrefix);
producer = std::make_unique<PartialProducer>(face, m_keyChain, 40, syncPrefix, userPrefix);
addUserNodes("testUser", 10);
}
@@ -51,14 +53,13 @@ public:
void
addConsumer(int id, const std::vector<std::string>& subscribeTo, bool linkToProducer = true)
{
consumerFaces[id] =
std::make_shared<util::DummyClientFace>(m_io, util::DummyClientFace::Options{true, true});
consumerFaces[id] = std::make_unique<ndn::DummyClientFace>(m_io, m_keyChain,
ndn::DummyClientFace::Options{true, true});
if (linkToProducer) {
face.linkTo(*consumerFaces[id]);
}
consumers[id] = std::make_shared<Consumer>(syncPrefix, *consumerFaces[id],
consumers[id] = std::make_unique<Consumer>(syncPrefix, *consumerFaces[id],
[&, id] (const auto& availableSubs) {
numHelloDataRcvd++;
BOOST_CHECK(checkSubList(availableSubs));
@@ -112,7 +113,7 @@ public:
{
// zeroth is added through constructor
for (int i = 1; i < numOfUserNodes; i++) {
producer->addUserNode(prefix + "-" + to_string(i));
producer->addUserNode(prefix + "-" + std::to_string(i));
}
}
@@ -131,15 +132,16 @@ public:
producer->updateSeqNo(prefix, seq);
}
util::DummyClientFace face{m_io, {true, true}};
Name syncPrefix{"psync"};
Name userPrefix{"testUser-0"};
protected:
ndn::DummyClientFace face{m_io, m_keyChain, {true, true}};
const Name syncPrefix{"psync"};
const Name userPrefix{"testUser-0"};
shared_ptr<PartialProducer> producer;
std::unique_ptr<PartialProducer> producer;
std::map<Name, uint64_t> oldSeqMap;
shared_ptr<Consumer> consumers[3];
shared_ptr<util::DummyClientFace> consumerFaces[3];
std::array<std::unique_ptr<Consumer>, 3> consumers;
std::array<std::unique_ptr<ndn::DummyClientFace>, 3> consumerFaces;
int numHelloDataRcvd = 0;
int numSyncDataRcvd = 0;
};
@@ -288,10 +290,10 @@ BOOST_AUTO_TEST_CASE(ReplicatedProducer)
// Link to first producer goes down
face.unlink();
util::DummyClientFace face2(m_io, {true, true});
PartialProducer replicatedProducer(40, face2, syncPrefix, userPrefix);
ndn::DummyClientFace face2(m_io, m_keyChain, {true, true});
PartialProducer replicatedProducer(face2, m_keyChain, 40, syncPrefix, userPrefix);
for (int i = 1; i < 10; i++) {
replicatedProducer.addUserNode("testUser-" + to_string(i));
replicatedProducer.addUserNode("testUser-" + std::to_string(i));
}
advanceClocks(ndn::time::milliseconds(10));
replicatedProducer.publishName("testUser-2");
@@ -326,7 +328,7 @@ BOOST_AUTO_TEST_CASE(ApplicationNack)
oldSeqMap = producer->m_prefixes;
for (int i = 0; i < 50; i++) {
Name prefix("testUser-" + to_string(i));
Name prefix("testUser-" + std::to_string(i));
producer->updateSeqNo(prefix, producer->getSeqNo(prefix).value() + 1);
}
// Next sync interest should trigger the nack
@@ -338,7 +340,7 @@ BOOST_AUTO_TEST_CASE(ApplicationNack)
bool nackRcvd = false;
for (const auto& data : face.sentData) {
if (data.getContentType() == tlv::ContentType_Nack) {
if (data.getContentType() == ndn::tlv::ContentType_Nack) {
nackRcvd = true;
break;
}
@@ -387,7 +389,7 @@ BOOST_AUTO_TEST_CASE(SegmentedSync)
std::vector<std::string> subscribeTo;
for (int i = 1; i < 10; i++) {
subscribeTo.push_back(longNameToExceedDataSize.toUri() + "-" + to_string(i));
subscribeTo.push_back(longNameToExceedDataSize.toUri() + "-" + std::to_string(i));
}
addConsumer(0, subscribeTo);
@@ -404,7 +406,7 @@ BOOST_AUTO_TEST_CASE(SegmentedSync)
oldSeqMap = producer->m_prefixes;
for (int i = 1; i < 10; i++) {
producer->updateSeqNo(longNameToExceedDataSize.toUri() + "-" + to_string(i), 1);
producer->updateSeqNo(longNameToExceedDataSize.toUri() + "-" + std::to_string(i), 1);
}
advanceClocks(ndn::time::milliseconds(999));
+20 -23
View File
@@ -1,6 +1,6 @@
/* -*- Mode:C++; c-file-style:"gnu"; indent-tabs-mode:nil; -*- */
/*
* Copyright (c) 2014-2020, The University of Memphis
* Copyright (c) 2014-2023, The University of Memphis
*
* This file is part of PSync.
* See AUTHORS.md for complete list of PSync authors and contributors.
@@ -20,63 +20,60 @@
#include "PSync/producer-base.hpp"
#include "tests/boost-test.hpp"
#include "tests/key-chain-fixture.hpp"
#include <ndn-cxx/data.hpp>
#include <ndn-cxx/interest.hpp>
#include <ndn-cxx/util/dummy-client-face.hpp>
namespace psync {
using namespace ndn;
using ndn::Name;
BOOST_AUTO_TEST_SUITE(TestProducerBase)
BOOST_AUTO_TEST_CASE(Constructor)
class ProducerBaseFixture : public tests::KeyChainFixture
{
util::DummyClientFace face;
BOOST_REQUIRE_NO_THROW(ProducerBase(40, face, Name("/psync"), Name("/testUser")));
}
protected:
ndn::DummyClientFace m_face;
};
BOOST_FIXTURE_TEST_SUITE(TestProducerBase, ProducerBaseFixture)
BOOST_AUTO_TEST_CASE(Basic)
{
util::DummyClientFace face;
Name userNode("/testUser");
ProducerBase producerBase(40, face, Name("/psync"), userNode);
ProducerBase producerBase(m_face, m_keyChain, 40, Name("/psync"), userNode);
// Hash table size should be 40 + 40/2 = 60 (which is perfectly divisible by N_HASH = 3)
BOOST_CHECK_EQUAL(producerBase.m_iblt.getHashTable().size(), 60);
BOOST_CHECK_EQUAL(producerBase.getSeqNo(userNode).value(), 0);
producerBase.updateSeqNo(userNode, 1);
BOOST_CHECK(producerBase.getSeqNo(userNode.toUri()).value() == 1);
BOOST_CHECK(producerBase.getSeqNo(userNode).value() == 1);
std::string prefixWithSeq = Name(userNode).appendNumber(1).toUri();
auto prefixWithSeq = Name(userNode).appendNumber(1);
uint32_t hash = producerBase.m_biMap.right.find(prefixWithSeq)->second;
Name prefix(producerBase.m_biMap.left.find(hash)->second);
BOOST_CHECK_EQUAL(prefix.getPrefix(-1), userNode);
producerBase.removeUserNode(userNode);
BOOST_CHECK(producerBase.getSeqNo(userNode.toUri()) == nullopt);
BOOST_CHECK(producerBase.getSeqNo(userNode) == std::nullopt);
BOOST_CHECK(producerBase.m_biMap.right.find(prefixWithSeq) == producerBase.m_biMap.right.end());
BOOST_CHECK(producerBase.m_biMap.left.find(hash) == producerBase.m_biMap.left.end());
Name nonExistentUserNode("/notAUser");
producerBase.updateSeqNo(nonExistentUserNode, 1);
BOOST_CHECK(producerBase.m_biMap.right.find(Name(nonExistentUserNode).appendNumber(1).toUri()) ==
BOOST_CHECK(producerBase.m_biMap.right.find(Name(nonExistentUserNode).appendNumber(1)) ==
producerBase.m_biMap.right.end());
}
BOOST_AUTO_TEST_CASE(ApplicationNack)
{
util::DummyClientFace face;
ProducerBase producerBase(40, face, Name("/psync"), Name("/testUser"));
ProducerBase producerBase(m_face, m_keyChain, 40, Name("/psync"), Name("/testUser"));
BOOST_CHECK_EQUAL(face.sentData.size(), 0);
producerBase.m_syncReplyFreshness = 1_s;
BOOST_CHECK_EQUAL(m_face.sentData.size(), 0);
producerBase.sendApplicationNack(Name("test"));
face.processEvents(10_ms);
m_face.processEvents(10_ms);
BOOST_REQUIRE_EQUAL(face.sentData.size(), 1);
BOOST_CHECK_EQUAL(face.sentData.front().getContentType(), tlv::ContentType_Nack);
BOOST_REQUIRE_EQUAL(m_face.sentData.size(), 1);
BOOST_CHECK_EQUAL(m_face.sentData.front().getContentType(), ndn::tlv::ContentType_Nack);
}
BOOST_AUTO_TEST_SUITE_END()
+43 -26
View File
@@ -1,6 +1,6 @@
/* -*- Mode:C++; c-file-style:"gnu"; indent-tabs-mode:nil; -*- */
/*
* Copyright (c) 2014-2020, The University of Memphis
* Copyright (c) 2014-2023, The University of Memphis
*
* This file is part of PSync.
* See AUTHORS.md for complete list of PSync authors and contributors.
@@ -22,27 +22,26 @@
#include "tests/boost-test.hpp"
#include "tests/io-fixture.hpp"
#include "tests/key-chain-fixture.hpp"
#include <ndn-cxx/data.hpp>
#include <ndn-cxx/interest.hpp>
#include <ndn-cxx/security/validator-null.hpp>
#include <ndn-cxx/util/dummy-client-face.hpp>
#include <ndn-cxx/util/segment-fetcher.hpp>
namespace psync {
using namespace ndn;
using namespace ndn::time_literals;
using ndn::Interest;
using ndn::Name;
class SegmentPublisherFixture : public tests::IoFixture
class SegmentPublisherFixture : public tests::IoFixture, public tests::KeyChainFixture
{
public:
protected:
SegmentPublisherFixture()
: face(m_io, util::DummyClientFace::Options{true, true})
, publisher(face, keyChain)
{
face.setInterestFilter(InterestFilter("/hello/world"),
bind(&SegmentPublisherFixture::onInterest, this, _1, _2),
[] (auto&&...) { BOOST_CHECK(false); });
m_face.setInterestFilter(ndn::InterestFilter("/hello/world"),
bind(&SegmentPublisherFixture::onInterest, this, _2),
[] (auto&&...) { BOOST_CHECK(false); });
advanceClocks(10_ms);
for (int i = 0; i < 1000; ++i) {
@@ -58,7 +57,7 @@ public:
void
expressInterest(const Interest& interest)
{
fetcher = util::SegmentFetcher::start(face, interest, security::getAcceptAllValidator());
fetcher = ndn::SegmentFetcher::start(m_face, interest, ndn::security::getAcceptAllValidator());
fetcher->onComplete.connect([this] (auto&&...) { numComplete++; });
fetcher->onError.connect([] (auto&&...) { BOOST_CHECK(false); });
@@ -66,7 +65,7 @@ public:
}
void
onInterest(const Name& prefix, const Interest& interest)
onInterest(const Interest& interest)
{
if (publisher.replyFromStore(interest.getName())) {
numRepliesFromStore++;
@@ -82,16 +81,17 @@ public:
}
}
util::DummyClientFace face;
KeyChain keyChain;
SegmentPublisher publisher;
shared_ptr<util::SegmentFetcher> fetcher;
protected:
ndn::DummyClientFace m_face{m_io, m_keyChain, {true, true}};
SegmentPublisher publisher{m_face, m_keyChain};
std::shared_ptr<ndn::SegmentFetcher> fetcher;
Name dataName;
time::milliseconds freshness = 1_s;
detail::State state;
int numComplete = 0;
int numRepliesFromStore = 0;
static constexpr ndn::time::milliseconds freshness = 1_s;
};
BOOST_FIXTURE_TEST_SUITE(TestSegmentPublisher, SegmentPublisherFixture)
@@ -99,6 +99,7 @@ BOOST_FIXTURE_TEST_SUITE(TestSegmentPublisher, SegmentPublisherFixture)
BOOST_AUTO_TEST_CASE(Basic)
{
BOOST_CHECK_EQUAL(publisher.m_ims.size(), 0);
expressInterest(Interest("/hello/world"));
BOOST_CHECK_EQUAL(numComplete, 1);
// First segment is answered directly in publish,
@@ -106,12 +107,20 @@ BOOST_AUTO_TEST_CASE(Basic)
BOOST_CHECK_EQUAL(numRepliesFromStore, 2);
BOOST_CHECK_EQUAL(publisher.m_ims.size(), 3);
for (const auto& data : publisher.m_ims) {
BOOST_TEST_CONTEXT(data.getName()) {
BOOST_REQUIRE_EQUAL(data.getName().size(), 4);
BOOST_CHECK(data.getName()[-1].isSegment());
BOOST_CHECK(data.getName()[-2].isVersion());
}
}
numRepliesFromStore = 0;
expressInterest(Interest("/hello/world"));
BOOST_CHECK_EQUAL(numComplete, 2);
BOOST_CHECK_EQUAL(numRepliesFromStore, 3);
advanceClocks(time::milliseconds(freshness));
advanceClocks(freshness);
BOOST_CHECK_EQUAL(publisher.m_ims.size(), 0);
numRepliesFromStore = 0;
@@ -120,19 +129,19 @@ BOOST_AUTO_TEST_CASE(Basic)
BOOST_CHECK_EQUAL(numRepliesFromStore, 2);
numRepliesFromStore = 0;
face.expressInterest(Interest("/hello/world/").setCanBePrefix(true),
[this] (auto&&...) { this->numComplete++; },
[] (auto&&...) { BOOST_CHECK(false); },
[] (auto&&...) { BOOST_CHECK(false); });
m_face.expressInterest(Interest("/hello/world").setCanBePrefix(true),
[this] (auto&&...) { this->numComplete++; },
[] (auto&&...) { BOOST_CHECK(false); },
[] (auto&&...) { BOOST_CHECK(false); });
advanceClocks(10_ms);
BOOST_CHECK_EQUAL(numComplete, 4);
BOOST_CHECK_EQUAL(numRepliesFromStore, 1);
}
BOOST_AUTO_TEST_CASE(LargerDataName)
BOOST_AUTO_TEST_CASE(LongerDataName)
{
BOOST_CHECK_EQUAL(publisher.m_ims.size(), 0);
dataName = Name("/hello/world/IBF");
BOOST_CHECK_EQUAL(publisher.m_ims.size(), 0);
expressInterest(Interest("/hello/world"));
BOOST_CHECK_EQUAL(numComplete, 1);
@@ -141,7 +150,15 @@ BOOST_AUTO_TEST_CASE(LargerDataName)
BOOST_CHECK_EQUAL(numRepliesFromStore, 2);
BOOST_CHECK_EQUAL(publisher.m_ims.size(), 3);
advanceClocks(time::milliseconds(freshness));
for (const auto& data : publisher.m_ims) {
BOOST_TEST_CONTEXT(data.getName()) {
BOOST_REQUIRE_EQUAL(data.getName().size(), 5);
BOOST_CHECK(data.getName()[-1].isSegment());
BOOST_CHECK(data.getName()[-2].isVersion());
}
}
advanceClocks(freshness);
BOOST_CHECK_EQUAL(publisher.m_ims.size(), 0);
}
+6 -6
View File
@@ -1,6 +1,6 @@
/* -*- Mode:C++; c-file-style:"gnu"; indent-tabs-mode:nil; -*- */
/*
* Copyright (c) 2014-2020, The University of Memphis
* Copyright (c) 2014-2022, The University of Memphis
*
* This file is part of PSync.
* See AUTHORS.md for complete list of PSync authors and contributors.
@@ -56,14 +56,14 @@ BOOST_AUTO_TEST_CASE(Compression)
const uint8_t uncompressed[] = {'t', 'e', 's', 't'};
for (const auto& s : supported) {
BOOST_CHECK_NO_THROW(compress(s, uncompressed, sizeof(uncompressed)));
auto compressed = compress(s, uncompressed, sizeof(uncompressed));
BOOST_CHECK_NO_THROW(decompress(s, compressed->data(), compressed->size()));
BOOST_CHECK_NO_THROW(compress(s, uncompressed));
auto compressed = compress(s, uncompressed);
BOOST_CHECK_NO_THROW(decompress(s, *compressed));
}
for (const auto& s : notSupported) {
BOOST_CHECK_THROW(compress(s, uncompressed, sizeof(uncompressed)), CompressionError);
BOOST_CHECK_THROW(decompress(s, uncompressed, sizeof(uncompressed)), CompressionError);
BOOST_CHECK_THROW(compress(s, uncompressed), CompressionError);
BOOST_CHECK_THROW(decompress(s, uncompressed), CompressionError);
}
}
+8 -5
View File
@@ -1,8 +1,11 @@
# -*- Mode: python; py-indent-offset: 4; indent-tabs-mode: nil; coding: utf-8; -*-
top = '..'
def build(bld):
bld.program(target='../unit-tests',
name='unit-tests',
source=bld.path.ant_glob('**/*.cpp'),
use='PSync',
install_path=None)
bld.program(
target=f'{top}/unit-tests',
name='unit-tests',
source=bld.path.ant_glob('**/*.cpp'),
use='BOOST_TESTS PSync',
install_path=None)
Vendored
+8 -8
View File
File diff suppressed because one or more lines are too long
+46 -33
View File
@@ -1,9 +1,10 @@
# -*- Mode: python; py-indent-offset: 4; indent-tabs-mode: nil; coding: utf-8; -*-
from waflib import Context, Logs, Utils
import os, subprocess
import os
import subprocess
from waflib import Context, Logs
VERSION = '0.3.0'
VERSION = '0.4.0'
APPNAME = 'PSync'
GIT_TAG_PREFIX = ''
@@ -11,7 +12,6 @@ BOOST_COMPRESSION_CODE = '''
#include <boost/iostreams/filter/{0}.hpp>
int main() {{ boost::iostreams::{0}_compressor test; }}
'''
COMPRESSION_SCHEMES = ['zlib', 'gzip', 'bzip2', 'lzma', 'zstd']
def options(opt):
@@ -28,8 +28,8 @@ def options(opt):
help='Build unit tests')
for scheme in COMPRESSION_SCHEMES:
optgrp.add_option('--without-{}'.format(scheme), action='store_true', default=False,
help='Build without {}'.format(scheme))
optgrp.add_option(f'--without-{scheme}', action='store_true', default=False,
help=f'Disable support for {scheme} (de)compression')
def configure(conf):
conf.load(['compiler_cxx', 'gnu_dirs',
@@ -39,22 +39,33 @@ def configure(conf):
conf.env.WITH_EXAMPLES = conf.options.with_examples
conf.env.WITH_TESTS = conf.options.with_tests
conf.check_cfg(package='libndn-cxx', args=['--cflags', '--libs'], uselib_store='NDN_CXX',
pkg_config_path=os.environ.get('PKG_CONFIG_PATH', '%s/pkgconfig' % conf.env.LIBDIR))
conf.find_program('dot', mandatory=False)
boost_libs = ['system', 'iostreams']
if conf.env.WITH_TESTS:
boost_libs.append('unit_test_framework')
# Prefer pkgconf if it's installed, because it gives more correct results
# on Fedora/CentOS/RHEL/etc. See https://bugzilla.redhat.com/show_bug.cgi?id=1953348
# Store the result in env.PKGCONFIG, which is the variable used inside check_cfg()
conf.find_program(['pkgconf', 'pkg-config'], var='PKGCONFIG')
conf.check_boost(lib=boost_libs, mt=True)
pkg_config_path = os.environ.get('PKG_CONFIG_PATH', f'{conf.env.LIBDIR}/pkgconfig')
conf.check_cfg(package='libndn-cxx', args=['libndn-cxx >= 0.8.1', '--cflags', '--libs'],
uselib_store='NDN_CXX', pkg_config_path=pkg_config_path)
conf.check_boost(lib='iostreams', mt=True)
if conf.env.BOOST_VERSION_NUMBER < 107100:
conf.fatal('The minimum supported version of Boost is 1.71.0.\n'
'Please upgrade your distribution or manually install a newer version of Boost.\n'
'For more information, see https://redmine.named-data.net/projects/nfd/wiki/Boost')
for scheme in COMPRESSION_SCHEMES:
if getattr(conf.options, 'without_{}'.format(scheme)):
if getattr(conf.options, f'without_{scheme}'):
continue
conf.check_cxx(fragment=BOOST_COMPRESSION_CODE.format(scheme),
use='BOOST', execute=False, mandatory=False,
msg='Checking for {} support in boost iostreams'.format(scheme),
define_name='HAVE_{}'.format(scheme.upper()))
msg=f'Checking for {scheme} support in boost iostreams',
define_name=f'HAVE_{scheme.upper()}')
if conf.env.WITH_TESTS:
conf.check_boost(lib='unit_test_framework', mt=True, uselib_store='BOOST_TESTS')
conf.check_compiler_flags()
@@ -75,13 +86,14 @@ def configure(conf):
conf.write_config_header('PSync/detail/config.hpp', define_prefix='PSYNC_')
def build(bld):
bld.shlib(target='PSync',
vnum=VERSION,
cnum=VERSION,
source=bld.path.ant_glob('PSync/**/*.cpp'),
use='NDN_CXX BOOST',
includes='.',
export_includes='.')
bld.shlib(
target='PSync',
vnum=VERSION,
cnum=VERSION,
source=bld.path.ant_glob('PSync/**/*.cpp'),
use='BOOST NDN_CXX',
includes='.',
export_includes='.')
if bld.env.WITH_TESTS:
bld.recurse('tests')
@@ -89,12 +101,10 @@ def build(bld):
if bld.env.WITH_EXAMPLES:
bld.recurse('examples')
# Install header files
headers = bld.path.ant_glob('PSync/**/*.hpp')
bld.install_files(bld.env.INCLUDEDIR, headers,
relative_trick=True)
bld.install_files('${INCLUDEDIR}/PSync/detail',
bld.path.find_resource('PSync/detail/config.hpp'))
bld.install_files('${INCLUDEDIR}', headers, relative_trick=True)
bld.install_files('${INCLUDEDIR}/PSync/detail', 'PSync/detail/config.hpp')
bld(features='subst',
source='PSync.pc.in',
@@ -119,6 +129,7 @@ def doxygen(bld):
target=['docs/doxygen.conf',
'docs/named_data_theme/named_data_footer-with-analytics.html'],
VERSION=VERSION,
HAVE_DOT='YES' if bld.env.DOT else 'NO',
HTML_FOOTER='../build/docs/named_data_theme/named_data_footer-with-analytics.html' \
if os.getenv('GOOGLE_ANALYTICS', None) \
else '../docs/named_data_theme/named_data_footer.html',
@@ -152,16 +163,16 @@ def version(ctx):
# first, try to get a version string from git
gotVersionFromGit = False
try:
cmd = ['git', 'describe', '--always', '--match', '%s*' % GIT_TAG_PREFIX]
out = subprocess.check_output(cmd, universal_newlines=True).strip()
cmd = ['git', 'describe', '--always', '--match', f'{GIT_TAG_PREFIX}*']
out = subprocess.run(cmd, capture_output=True, check=True, text=True).stdout.strip()
if out:
gotVersionFromGit = True
if out.startswith(GIT_TAG_PREFIX):
Context.g_module.VERSION = out.lstrip(GIT_TAG_PREFIX)
else:
# no tags matched
Context.g_module.VERSION = '%s-commit-%s' % (VERSION_BASE, out)
except (OSError, subprocess.CalledProcessError):
Context.g_module.VERSION = f'{VERSION_BASE}-commit-{out}'
except (OSError, subprocess.SubprocessError):
pass
versionFile = ctx.path.find_node('VERSION.info')
@@ -179,17 +190,19 @@ def version(ctx):
# already up-to-date
return
except EnvironmentError as e:
Logs.warn('%s exists but is not readable (%s)' % (versionFile, e.strerror))
Logs.warn(f'{versionFile} exists but is not readable ({e.strerror})')
else:
versionFile = ctx.path.make_node('VERSION.info')
try:
versionFile.write(Context.g_module.VERSION)
except EnvironmentError as e:
Logs.warn('%s is not writable (%s)' % (versionFile, e.strerror))
Logs.warn(f'{versionFile} is not writable ({e.strerror})')
def dist(ctx):
ctx.algo = 'tar.xz'
version(ctx)
def distcheck(ctx):
ctx.algo = 'tar.xz'
version(ctx)