emdee
/
toxygen_wrapper
1
0
Fork 0
Code Issues Pull Requests Actions Projects Releases Wiki Activity

update
Gitea Actions Demo / Explore-Gitea-Actions (push) Has been cancelled Details

This commit is contained in:
emdee@spm.plastiras.org 2024-02-19 12:45:23 +00:00
parent 11f8169e2b
commit 6e3dd26dad
8 changed files with 36 additions and 1858 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

4
Makefile
View File

@ -27,12 +27,12 @@ help::
PYTHONPATH=$${PWD}/src \
$(PYTHON) src/toxygen_wrapper/tests/tests_wrapper.py --help
test::
test:: test_proxy
test_direct::
cp -p ${HOME}/.config/tox/DHTnodes.json /tmp/toxygen_nodes.json||true
PYTHONPATH=$${PWD}/src \
TOR_CONTROLLER_PASSWORD=${PASS} \
sudo -u bin $(PYTHON_EXE_MSYS) src/toxygen_wrapper/tests/tests_wrapper.py \
$(PYTHON_EXE_MSYS) src/toxygen_wrapper/tests/tests_wrapper.py \
--norequest=True \
--socket_timeout=10.0 \
--test_timeout=${iTEST_TIMEOUT} \

24
README.md
View File

@ -16,7 +16,7 @@ and the list of still unwrapped calls as of Sept. 2022 can be found in
It has been tested with UDP and TCP proxy (Tor). It has ***not*** been
tested on Windows, and there may be some minor breakage, which should be
easy to fix. There is a good coverage integration testsuite in ```tox_wrapper/tests```.
easy to fix. There is a good coverage integration testsuite in ```toxygen_wrapper/tests```.
Change to that directory and run ```tests_wrapper.py --help```; the test
suite gives a good set of examples of usage.
@ -26,18 +26,18 @@ Run ```python3 setup.py install``` or put the parent of the wrapper
directory on your PYTHONPATH and touch a file called `__init__.py`
in its parent directory.
Then you need a ```libs``` directory beside the ```tox_wrapper``` directory
Then you need a ```libs``` directory beside the ```toxygen_wrapper``` directory
and you need to link your ```libtoxcore.so``` and ```libtoxav.so```
and ```libtoxencryptsave.so``` into it. Link all 3 filenames
to ```libtoxcore.so``` if you have only ```libtoxcore.so```
(which is usually the case if you built ```c-toxcore``` with ```cmake```
rather than ```autogen/configure```). If you want to be different,
the environment variable TOXCORE_LIBS overrides the location of ```libs```;
look in the file ```tox_wrapper/libtox.py``` for the details.
look in the file ```toxygen_wrapper/libtox.py``` for the details.
# Tests
To test, run ```python3 tox_wrapper/tests/tests_wrapper.py --help```
To test, run ```python3 toxygen_wrapper/tests/tests_wrapper.py --help```
As is, the code in ```tox.py``` is very verbose. Edit the file to change
```
@ -50,7 +50,7 @@ def LOG_TRACE(a): pass # print('TRAC> '+a)
to all ```pass #``` or use ```logging.logger``` to suite your tastes.
```logging.logger``` can be dangerous in callbacks in ```Qt``` applications,
so we use simple print statements as default. The same applies to
```tox_wrapper/tests/tests_wrapper.py```.
```toxygen_wrapper/tests/tests_wrapper.py```.
## Prerequisites
@ -94,14 +94,16 @@ debugger is a crucial advantage.
Although Tox works over Tor, we do not recommend its usage for
anonymity as it leaks DNS requests due to a 6-year old known security
issue: https://github.com/TokTok/c-toxcore/issues/469 unless your Tox client
does hostname lookups before calling Tox (like toxygen does). Otherwise, do not
use it for anonymous communication unless you have a TCP and UDP firewall in place.
does hostname lookups before calling Tox (like toxygen does). Otherwise,
do not use it for anonymous communication unless you have a firewall in place.
The Tox project does not follow semantic versioning so the project may break the
underlying ctypes wrapper at any time; it's not possible to use Tox version numbers
to tell what the API will be. In which case you'll have to go into the tox.py file in
The Tox project does not follow semantic versioning of its main structures
so the project may break the underlying ctypes wrapper at any time;
it's not possible to use Tox version numbers to tell what the API will be.
In which case you'll have to go into the tox.py file in
https://git.plastiras.org/emdee/toxygen_wrapper to fix it yourself.
The last tested git commit is 5dd9ee3f65423a4095cacb8396a5d406a27610c7 2024-02-10
The last tested git commit is 5dd9ee3f65423a4095cacb8396a5d406a27610c7
2024-02-10
Up-to-date code is on https://git.plastiras.org/emdee/toxygen_wrapper

2
src/toxygen_wrapper/tests/support_testing.py
View File

@ -635,7 +635,7 @@ def generate_nodes(oArgs,
nodes_count=nodes_count,
ipv=ipv,
udp_not_tcp=udp_not_tcp)
assert lNodes
assert lNodes, f"No nodes from {sFile}"
aNODES[sKey] = lNodes
return aNODES[sKey]

11
src/toxygen_wrapper/tests/tests_wrapper.py
View File

@ -98,11 +98,11 @@ from toxygen_wrapper.toxcore_enums_and_consts import (TOX_ADDRESS_SIZE,
try:
import support_testing as ts
import support_onions as so
from wrapper_mixin import WrapperMixin
except ImportError:
import toxygen_wrapper.tests.support_testing as ts
import toxygen_wrapper.tests.support_onions as so
from wrapper_mixin import WrapperMixin
from tox_wrapper.wrapper_mixin import WrapperMixin
try:
from tests.toxygen_tests import test_sound_notification
@ -132,7 +132,6 @@ else:
ADDR_SIZE = 38 * 2
CLIENT_ID_SIZE = 32 * 2
THRESHOLD = 120 # >25
fSOCKET_TIMEOUT = 15.0
iN = 6
@ -1261,6 +1260,8 @@ class ToxSuite(unittest.TestCase, WrapperMixin):
if hasattr(self, 'abid') and self.abid >= 0:
self.alice.friend_delete(self.abid)
@unittest.skipIf( not os.environ.get('TOR_CONTROLLER_PASSWORD', ''), \
"Requires tor controller password")
def test_onion_intros(self) -> None: # timeout intermittently
# introduction points are needed for onion services
if self.bob._args.proxy_type == 2:
@ -1649,6 +1650,10 @@ def oTestsToxOptions(oArgs):
def oArgparse(lArgv):
global THRESHOLD
if os.environ.get('socks_proxy'):
THRESHOLD = 150 # >25
else:
THRESHOLD = 30 # >25
parser = ts.oMainArgparser()
parser.add_argument('--norequest',type=str, default='False',
choices=['True','False'],

21
src/toxygen_wrapper/tests/wrapper_mixin.py
View File

@ -299,10 +299,11 @@ class WrapperMixin():
def bob_add_alice_as_friend_norequest(self) -> bool:
if not self.bBobNeedAlice(): return True
iRet = self.bob.friend_add_norequest(self.alice._address)
apk = self.alice.self_get_public_key()
iRet = self.bob.friend_add_norequest(apk)
if iRet < 0:
return False
self.baid = self.bob.friend_by_public_key(self.alice._address)
self.baid = self.bob.friend_by_public_key(apk)
assert self.baid >= 0, self.baid
assert self.bob.friend_exists(self.baid), "bob.friend_exists"
assert not self.bob.friend_exists(self.baid + 1)
@ -313,10 +314,11 @@ class WrapperMixin():
def alice_add_bob_as_friend_norequest(self) -> bool:
if not self.bAliceNeedAddBob(): return True
iRet = self.alice.friend_add_norequest(self.bob._address)
bpk = self.bob.self_get_public_key()
iRet = self.alice.friend_add_norequest(bpk)
if iRet < 0:
return False
self.abid = self.alice.friend_by_public_key(self.bob._address)
self.abid = self.alice.friend_by_public_key(bpk)
assert self.abid >= 0, self.abid
assert self.abid in self.alice.self_get_friend_list()
assert self.alice.friend_exists(self.abid), "alice.friend_exists"
@ -376,6 +378,7 @@ class WrapperMixin():
setattr(self.bob, sSlot, True)
setattr(self.bob, sSlot, None)
apk = self.alice.self_get_public_key()
inum = -1
try:
inum = self.bob.friend_add(self.alice._address, bytes(MSG, 'UTF-8'))
@ -387,8 +390,8 @@ class WrapperMixin():
self.alice.callback_friend_request(alices_on_friend_request)
if not self.wait_otox_attrs(self.bob, [sSlot]):
LOG_WARN(f"bob_add_alice_as_friend NO setting {sSlot}")
# return False
self.baid = self.bob.friend_by_public_key(self.alice._address)
return False
self.baid = self.bob.friend_by_public_key(apk)
assert self.baid >= 0, self.baid
assert self.bob.friend_exists(self.baid)
assert not self.bob.friend_exists(self.baid + 1)
@ -427,6 +430,7 @@ class WrapperMixin():
LOG_INFO(f"bobs_on_friend_request: {sSlot} = True ")
setattr(self.alice, sSlot, None)
bpk = self.bob.self_get_public_key()
inum = -1
try:
inum = self.alice.friend_add(self.bob._address, bytes(MSG, 'UTF-8'))
@ -434,8 +438,8 @@ class WrapperMixin():
self.bob.callback_friend_request(bobs_on_friend_request)
if not self.wait_otox_attrs(self.alice, [sSlot]):
LOG_WARN(f"alice.friend_add NO wait {sSlot}")
#? return False
self.abid = self.alice.friend_by_public_key(self.bob._address)
return False
self.abid = self.alice.friend_by_public_key(bpk)
assert self.abid >= 0, self.abid
assert self.alice.friend_exists(self.abid), "not exists"
assert not self.alice.friend_exists(self.abid + 1), "exists +1"
@ -683,4 +687,3 @@ class WrapperMixin():
if hasattr(self.bob, sSlot+'_cb') and \
getattr(self.bob, sSlot+'_cb'):
LOG.warning(f"self.bob.{sSlot}_cb EXIST")

1471
tox_profile.py
View File

File diff suppressed because it is too large Load Diff

24
tox_profile_examples.bash
View File

@ -1,24 +0,0 @@
#!/bin/sh -e
# -*- mode: sh; fill-column: 75; tab-width: 8; coding: utf-8-unix -*-
# some examples of tox-profile usage
export PYTHONPATH=/mnt/o/var/local/src/toxygen_wrapper.git
TOX_HOME=$HOME/.config/tox
NMAP_CMD='sudo -u debian-tor nmap'
echo INFO: check the download json file
python3 tox_profile.py --command nodes --nodes check \
$TOX_HOME/DHTnodes.json.new \
2>&1 | tee /tmp/DHTnodes.json.log
echo INFO: get the tcp nodes/ports from the downloaded json file
python3 tox_profile.py --command nodes --nodes select_tcp \
--output /tmp/DHTnodes.json.tcp \
$TOX_HOME/DHTnodes.json.new
echo INFO: run ping/nmap on the tcp nodes/ports from the downloaded json file
python3 tox_profile.py --command nodes --nodes nmap_tcp \
--nmap_cmd $NMAP_CMD \
--output /tmp/DHTnodes.json.tcp.out \
/tmp/DHTnodes.json.tcp

337
tox_profile_test.bash
View File

@ -1,337 +0,0 @@
#!/bin/sh
# -*- mode: sh; fill-column: 75; tab-width: 8; coding: utf-8-unix -*-
# tox_profile.py has a lot of features so it needs test coverage
PREFIX=/mnt/o/var/local
ROLE=text
DEBUG=1
EXE=/var/local/bin/python3.bash
WRAPPER=$PREFIX/src/toxygen_wrapper.git
tox=$HOME/.config/tox/toxic_profile.tox
[ -s $tox ] || exit 2
target=$PREFIX/src/tox_profile/tox_profile.py
OUT=/tmp/toxic_profile
ps ax | grep -q tor && netstat -n4le | grep -q :9050
[ $? -eq 0 ] && HAVE_TOR=1 || HAVE_TOR=0
[ -f /usr/local/bin/usr_local_tput.bash ] && \
. /usr/local/bin/usr_local_tput.bash || {
DBUG() { echo DEBUG $* ; }
INFO() { echo INFO $* ; }
WARN() { echo WARN $* ; }
ERROR() { echo ERROR $* ; }
}
if [ -z "$TOXCORE_LIBS" ] && [ ! -d libs ] ; then
mkdir libs
cd libs
# /lib/x86_64-linux-gnu/libtoxcore.so.2
for pro in qtox toxic ; do
if which $pro 2> /dev/null ; then
DBUG linking to $pro libtoxcore
lib=$( ldd `which $pro` | grep libtoxcore|sed -e 's/.* => //' -e 's/ .*//')
[ -n "$lib" -a -f "$lib" ] || { WARN $Lib ; continue ; }
INFO linking to $lib
for elt in libtoxcore.so libtoxav.so libtoxencryptsave.so ; do
ln -s "$lib" "$elt"
done
export TOXCORE_LIBS=$PWD
break
fi
done
cd ..
elif [ -z "$TOXCORE_LIBS" ] && [ -d libs ] ; then
export TOXCORE_LIBS=$PWD/libs
fi
# set -- -e
[ -s $target ] || exit 1
[ -d $WRAPPER ] || {
ERROR wrapper is required https://git.plastiras.org/emdee/toxygen_wrapper
exit 3
}
export PYTHONPATH=$WRAPPER
json=$HOME/.config/tox/DHTnodes.json
[ -s $json ] || exit 4
which jq > /dev/null && HAVE_JQ=1 || HAVE_JQ=0
which nmap > /dev/null && HAVE_NMAP=1 || HAVE_NMAP=0
sudo rm -f $OUT.* /tmp/toxic_nodes.*
test_jq () {
[ $# -eq 3 ] || {
ERROR test_jq '#' "$@"
return 3
}
in=$1
out=$2
err=$3
[ -s $in ] || {
ERROR $i test_jq null $in
return 4
}
jq . < $in >$out 2>$err || {
ERROR $i test_jq $json
return 5
}
grep error: $err && {
ERROR $i test_jq $json
return 6
}
[ -s $out ] || {
ERROR $i null $out
return 7
}
[ -s $err ] || rm -f $err
return 0
}
i=0
[ "$HAVE_JQ" = 0 ] || \
test_jq $json /tmp/toxic_nodes.json /tmp/toxic_nodes.err || {
ERROR test_jq failed on $json
exit ${i}$?
}
[ -f /tmp/toxic_nodes.json ] || cp -p $json /tmp/toxic_nodes.json
json=/tmp/toxic_nodes.json
i=1
# required password
INFO $i decrypt $OUT.bin
$EXE $target --command decrypt --output $OUT.bin $tox || exit ${i}1
[ -s $OUT.bin ] || exit ${i}2
tox=$OUT.bin
INFO $i info $tox
$EXE $target --command info --info info $tox 2>$OUT.info || {
ERROR $i $EXE $target --command info --info info $tox
exit ${i}3
}
[ -s $OUT.info ] || exit ${i}4
INFO $i $EXE $target --command info --info save --output $OUT.save $tox
$EXE $target --command info --info save --output $OUT.save $tox 2>/dev/null || {
ERROR $?
exit ${i}5
}
[ -s $OUT.save ] || exit ${i}6
i=2
[ $# -ne 0 -a $1 -ne $i ] || \
! INFO $i Info and editing || \
for the_tox in $tox $OUT.save ; do
DBUG $i $the_tox
the_base=`echo $the_tox | sed -e 's/.save$//' -e 's/.tox$//'`
for elt in json yaml pprint repr ; do
if [ $elt = yaml -o $elt = json ] ; then
# ModuleNotFoundError
python3 -c "import $elt" 2>/dev/null || continue
fi
INFO $i $the_base.$elt
DBUG $EXE $target \
--command info --info $elt \
--output $the_base.$elt $the_tox '2>'$the_base.$elt.err
$EXE $target --command info --info $elt \
--output $the_base.$elt $the_tox 2>$the_base.$elt.err || {
tail $the_base.$elt.err
if [ $elt != yaml -a $elt != json ] ; then
exit ${i}0
else
WARN $elt
fi
}
[ -s $the_base.$elt ] || {
WARN no output $the_base.$elt
# exit ${i}1
}
done
DBUG $EXE $target --command edit --edit help $the_tox
$EXE $target --command edit --edit help $the_tox 2>/dev/null || exit ${i}2
# edit the status message
INFO $i $the_base.Status_message 'STATUSMESSAGE,.,Status_message,Toxxed on Toxic'
$EXE $target --command edit --edit 'STATUSMESSAGE,.,Status_message,Toxxed on Toxic' \
--output $the_base.Status_message.tox $the_tox 2>&1|grep EDIT || exit ${i}3
[ -s $the_base.Status_message.tox ] || exit ${i}3
$EXE $target --command info $the_base.Status_message.tox 2>&1|grep Toxxed || exit ${i}4
# edit the nick_name
INFO $i $the_base.Nick_name 'NAME,.,Nick_name,FooBar'
$EXE $target --command edit --edit 'NAME,.,Nick_name,FooBar' \
--output $the_base.Nick_name.tox $the_tox 2>&1|grep EDIT || exit ${i}5
[ -s $the_base.Nick_name.tox ] || exit ${i}5
$EXE $target --command info $the_base.Nick_name.tox 2>&1|grep FooBar || exit ${i}6
# set the DHTnodes to empty
INFO $i $the_base.noDHT 'DHT,.,DHTnode,'
$EXE $target --command edit --edit 'DHT,.,DHTnode,' \
--output $the_base.noDHT.tox $the_tox 2>&1|grep EDIT || exit ${i}7
[ -s $the_base.noDHT.tox ] || exit ${i}7
$EXE $target --command info $the_base.noDHT.tox 2>&1 | grep 'NO DHT' || exit ${i}8
done
i=3
[ "$#" -ne 0 -a "$1" != "$i" ] || \
[ "$HAVE_JQ" = 0 ] || \
! INFO $i Nodes || \
for the_json in $json ; do
DBUG $i $the_json
the_base=`echo $the_json | sed -e 's/.json$//' -e 's/.tox$//'`
for nmap in clean check select_tcp select_udp select_version; do
$EXE $target --command nodes --nodes $nmap \
--output $the_base.$nmap.json $the_json || {
WARN $i $the_json $nmap ${i}1
continue
}
[ -s $the_base.$nmap.json ] || {
WARN $i $the_json $nmap ${i}2
continue
}
[ $nmap = select_tcp ] && \
grep '"status_tcp": false' $the_base.$nmap.json && {
WARN $i $the_json $nmap ${i}3
continue
}
[ $nmap = select_udp ] && \
grep '"status_udp": false' $the_base.$nmap.json && {
WARN $i $the_json $nmap ${i}4
continue
}
test_jq $the_base.$nmap.json $the_base.$nmap.json.out /tmp/toxic_nodes.err || {
retval=$?
WARN $i $the_base.$nmap.json 3$?
}
INFO $i $the_base.$nmap
done
done
i=4
[ $# -ne 0 -a "$1" -ne $i ] || \
[ "$HAVE_TOR" = 0 ] || \
[ ! -f /etc/tor/torrc ] || \
! INFO $i Onions || \
for the_tox in /etc/tor/torrc ; do
DBUG $i $the_tox
the_base=`echo $OUT.save | sed -e 's/.save$//' -e 's/.tox$//'`
# exits
for slot in config test; do
if [ $slot = exits ] && ! netstat -nle4 | grep -q :9050 ; then
WARN Tor not running
continue
fi
INFO $target --command onions --onions $slot \
--output $the_base.$slot.out $the_tox
DBUG=1 $EXE $target --command onions --onions $slot \
--log_level 10 \
--output $the_base.$slot.out $the_tox|| {
WARN $i $?
continue
}
[ true -o -s $the_base.$slot.out ] || {
WARN $i empty $the_base.$slot.out
continue
}
done
done
# ls -l $OUT.* /tmp/toxic_nodes.*
# DEBUG=0 /usr/local/bin/proxy_ping_test.bash tor || exit 0
ip route | grep ^def || exit 0
i=5
the_tox=$tox
[ $# -ne 0 -a "$1" != "$i" ] || \
[ "$HAVE_JQ" = 0 ] || \
[ "$HAVE_NMAP" = 0 ] || \
! INFO $i Making dogfood || \
for the_tox in $tox $OUT.save ; do
DBUG $i $the_tox
the_base=`echo $the_tox | sed -e 's/.save$//' -e 's/.tox$//'`
for nmap in nmap_relay nmap_dht nmap_path ; do
# [ $nmap = select_tcp ] && continue
if [ $nmap = nmap_dht ] && [ $HAVE_TOR = 1 ] ; then
INFO skipping $nmap because HAVE_TOR
continue
fi
INFO $i $the_base.$nmap
DBUG $target --command info --info $nmap \
--output $the_base.$nmap.out $the_tox
$EXE $target --command info --info $nmap \
--output $the_base.$nmap.out $the_tox 2>$the_base.$nmap.err || {
# select_tcp may be empty and jq errors
# exit ${i}1
WARN $i $? $the_base.$nmap.err
tail $the_base.$nmap.err
continue
}
[ -s $the_base.$nmap.out ] || {
WARN $i empty $the_base.$nmap.out
continue
}
done
done
i=6
[ $# -ne 0 -a "$1" != "$i" ] || \
[ "$HAVE_JQ" = 0 ] || \
! INFO $i Eating dogfood || \
for the_json in $json ; do
DBUG $i $the_json
the_base=`echo $the_json | sed -e 's/.save$//' -e 's/.json$//'`
for nmap in nmap_tcp nmap_udp ; do
if [ $nmap = nmap_udp ] && [ $HAVE_TOR = 1 ] ; then
INFO skipping $nmap because HAVE_TOR
continue
fi
INFO $i $target --command nodes --nodes $nmap --output $the_base.$nmap
$EXE $target --command nodes --nodes $nmap \
--output $the_base.$nmap $the_json 2>$the_base.$nmap.err || {
WARN $i $the_json $nmap ${i}1
continue
}
[ -s $the_base.$nmap ] || {
ERROR $i $the_json $nmap ${i}2
exit ${i}2
}
done
done
i=7
DBUG $i
$EXE $target --command nodes --nodes download \
--output /tmp/toxic_nodes.new $json || {
ERROR $i $EXE $target --command nodes --nodes download $json
exit ${i}1
}
[ -s /tmp/toxic_nodes.new ] || exit ${i}4
INFO $i downloaded /tmp/toxic_nodes.new
json=/tmp/toxic_nodes.new
[ $# -ne 0 -a "$1" != "$i" ] || \
[ "$HAVE_JQ" = 0 ] || \
jq . < $json >/tmp/toxic_nodes.new.json 2>>/tmp/toxic_nodes.new.json.err || {
ERROR $i jq $json
exit ${i}2
}
INFO $i jq from /tmp/toxic_nodes.new.json
[ $# -ne 0 -a "$1" != "$i" ] || \
[ "$HAVE_JQ" = 0 ] || \
grep error: /tmp/toxic_nodes.new.json.err && {
ERROR $i jq $json
exit ${i}3
}
INFO $i no errors in /tmp/toxic_nodes.new.err
exit 0