Hey axel!
so, we've scratching our heads today, facing some MTU issues on our
shiny new libre-mesh network,
and after a few hours debugging, tcpdumping and discussing, we came to
these conclusions:
* the invented src-addr in the bmxOut tunnel makes it impossible for
hops along the path to return "Packet Too Big" to the originating node.
So, if a particular link has a smaller MTU than the first link (say, a
VPN is involved), the packet will be silently dropped.
A==1500==B---1400---C===1500===D
A wants to send a packet to a network announced by D; so it creates a
tunnel with D as destination, and a "D-derived" fake address as src,
that matches the bmxIn "catchall" tunnel peer-addr in D
Then sends a 1460 + 40 = 1500 bytes packet through that tunnel,
B cannot push that packet to C, then tries to send back a ICMPv6 PTB...
but the src-addr it finds in the encapsulation is not A, but instead the
"D-derived" fake addr. Then, the ICMPv6 PTB is lost and A can never find
out about the smaller MTU.
This fundamentally breaks PMTUD which is a bad idea in IPv6
to avoid this there are 3 options:
* set mtu=1280 on every bmxOut tunnel (yuck! :( )
* probe before establishing each tunnel, with the real src-addr so that
PMTUD can happen correctly, until it reaches the desired endpoint node.
Then, use discovered PMTU for new tunnel. (*downside*: this will only
work as long as path doesn't change to pass through some thinner link.
In that case, PMTU will not be rediscovered, and packets will be dropped
again.)
* use the current bmxIn "catchall" tunnels only for sending special bmx6
control packets, that ask for a symmetric tunnel.
i.e.
1) A sends to D (2001:db8::D) a packet (encapsulated with a fake
src-addr, to be catched by the catchall @ D) with content "I'm A and
this is my real address 2001:db8::A; please make a dedicated tunnel for me"
2) D gets that packet and creates a tunnel with "peer 2001:db8::A", then
sends back an ACK to A, again using "A-derived" fake-addr as src
3) A gets the ACK and creates the tunnel with "peer 2001:db8::D"
*** now both ends have a symmetric tunnel between them, with real src
and dest address ***
4) A finally sends the real payload through the symmetric tunnel, this
payload (may be bigger than 1280, say... 1450) will be encapsulated with
the real src-addr of A, so if any node in the path needs to send back a
Packet-too-big, will be able to, and PMTUD will happen correctly.
(at the cost of a full RTT latency before the first payload packet, but
with a reasonable tunnel expire time as it has currently, that shouldn't
be terrible)
back in April, i remember we discussed the idea of symmetric tunnels,
and you brought up this "control connection" idea which i'm simply
redescribing here.
But that discussion was in another context, more like a 'feature', and
finally didn't really solve the idea we had originally,
yet, this PMTUD issue was not taken into account AFAIR, so the
"symmetric tunnel" idea now becomes more like a bugfix (i.e. don't
create PMTUD blackholes)
what do you think?
Cheers!!
gui
Hi from Garraf (Barcelona). Here we are with Gioacchino installing Libre-mesh nodes around and we are going to train a local group to lead Libre-mesh. People here use, understand and install guifi.net infraestructure nodes (RouterOS, AirOS, AP-WDS and BGP in both 5GHz and 2.4GHz), and they don't know more. So I just start to write a Libre-mesh HowTo oriented to this kind of base knowledge. I'm writing it in spanish and later I'm going to translate it to english and catalan. It's going to become a first approach to concepts like OpenWRT, ad-hoc, BMX6 and BATMAN-adv (without miths) for Guifi.net people.
It is based on questions and answers, so, if you want to help, you can write some questions or try to write some answer. I'm doing it ehterpad here:
http://pad.marsupi.org/manual-de-libre-mesh-para-guiferos
Hi all!
Following the instructions [1] on your site I obtain a clean OpenWrt image, no LiMe packages inside, no bmx6, no batctl on the flashed antenna.
I noticed the presence of a broken link:
ls -l lime-build/
lrwxrwxrwx 1 ilario ilario 25 mar 15 15:00 files -> build/lime-packages/files
maybe this is the cause?
Running an old stile compilation worked [2].
[1] git clone https://github.com/libre-mesh/lime-build.git
cd lime-build
make T=ar71xx J=4
[2] cd lime-build/build/trunk
./scripts/feeds update -a
./scripts/feeds install -a
make menuconfig (and select LiMe packages)
make
[21:03:30] <g10h4ck> guii nicoechaniz how are you ?
[21:03:46] <g10h4ck> did you find the piece for server mounting ?
[21:06:16] <guii> nope :( it's carnival/holidays until wednesday
[21:06:31] <guii> ah, wait
[21:06:39] <guii> i think you didn't get the news:
[21:06:55] <guii> the server PSU / motherboard / CPU died
Hi all!
Talking with some german guys they were having fun on me, because i said
tagged vlan are not working on wdrXXXX switch it seems we are just ignorant
and we are missing some knowld on how switch works on openwrt they said to me
that they tried on wdr3600 and it works fine and pointed to me to this page
http://freifunk.in-kiel.de/wiki/Konfiguration/Eigenes_WLAN
I do not have tp-link with me but i'll implement that as soon as possible and
hope someone with tp-link in hand can help me testing
Hi all,
I have two devices (Ubiquiti PicoStation M2 and Ubiquiti NanoStation M5) in a very simple configuration: both attached to a Mikrotik RB250GS gigabit switch (latest 1.11 firmware) and both running the latest libre-mesh firmware.
>From dmesg of the two devices I can see some error form batman loop avoidance:
br-lan: received packet on bat0 with own address as source address
Do you think that could be a libre-mesh related problem?
This is from the PicoStation
[...]
[ 15.520000] IPv6: ADDRCONF(NETDEV_UP): eth0: link is not ready
[ 15.520000] device eth0 entered promiscuous mode
[ 15.540000] IPv6: ADDRCONF(NETDEV_UP): br-lan: link is not ready
[ 15.640000] device br-lan entered promiscuous mode
[ 15.640000] IPv6: ADDRCONF(NETDEV_UP): anygw: link is not ready
[ 15.690000] IPv6: ADDRCONF(NETDEV_UP): eth0.11: link is not ready
[ 15.840000] IPv6: ADDRCONF(NETDEV_UP): eth0.12: link is not ready
[ 16.060000] batman_adv: bat0: Adding interface: eth0.11
[ 16.060000] batman_adv: bat0: The MTU of interface eth0.11 is too small (1496) to handle the transport of batman-adv packets. Packets going over this interface will be fragmented on layer2 which could impact the performance. Setting the MTU to 1532 would solve the problem.
[ 16.090000] batman_adv: bat0: Interface activated: eth0.11
[ 16.100000] 8021q: adding VLAN 0 to HW filter on device bat0
[ 16.100000] device bat0 entered promiscuous mode
[ 16.110000] br-lan: port 2(baINBOXt0) entered forwarding state
[ 16.110000] br-lan: port 2(bat0) entered forwarding state
[ 16.220000] IPv6: ADDRCONF(NETDEV_CHANGE): br-lan: link becomes ready
[ 16.220000] IPv6: ADDRCONF(NETDEV_CHANGE): anygw: link becomes ready
[ 16.240000] br-lan: received packet on bat0 with own address as source address
[ 16.410000] eth0: link up (100Mbps/Full duplex)
[ 16.410000] br-lan: port 1(eth0) entered forwarding state
[ 16.420000] br-lan: port 1(eth0) entered forwarding state
[ 16.420000] IPv6: ADDRCONF(NETDEV_CHANGE): eth0: link becomes ready
[ 16.440000] IPv6: ADDRCONF(NETDEV_CHANGE): eth0.11: link becomes ready
[ 16.480000] IPv6: ADDRCONF(NETDEV_CHANGE): eth0.12: link becomes ready
[ 17.340000] batman_adv: bat0: bridge_loop_avoidance: Changing from: disabled to: enabled
[ 17.370000] batman_adv: bat0: distributed_arp_table: Changing from: enabled to: disabled
[ 18.110000] br-lan: port 2(bat0) entered forwarding state
[ 18.420000] br-lan: port 1(eth0) entered forwarding state
[ 20.350000] IPv6: ADDRCONF(NETDEV_UP): wlan0_ap: link is not ready
[ 20.370000] IPv6: ADDRCONF(NETDEV_UP): wlan0_ap.12: link is not ready
[ 23.510000] device wlan0_ap entered promiscuous mode
[ 23.530000] IPv6: ADDRCONF(NETDEV_UP): wlan0_ap.12: link is not ready
[ 23.540000] IPv6: ADDRCONF(NETDEV_UP): wlan0_ap: link is not ready
[ 23.560000] IPv6: ADDRCONF(NETDEV_CHANGE): wlan0_ap: link becomes ready
[ 23.560000] IPv6: ADDRCONF(NETDEV_CHANGE): wlan0_ap.12: link becomes ready
[ 23.580000] br-lan: port 3(wlan0_ap) entered forwarding state
[ 23.580000] br-lan: port 3(wlan0_ap) entered forwarding state
[ 25.580000] br-lan: port 3(wlan0_ap) entered forwarding state
[ 26.040000] br-lan: received packet on bat0 with own address as source address
[ 27.090000] IPv6: ADDRCONF(NETDEV_UP): wlan0_adhoc: link is not ready
[ 27.110000] wlan0_adhoc: Created IBSS using preconfigured BSSID ca:fe:00:c0:ff:ee
[ 27.110000] wlan0_adhoc: Creating new IBSS network, BSSID ca:fe:00:c0:ff:ee
[ 27.120000] IPv6: ADDRCONF(NETDEV_CHANGE): wlan0_adhoc: link becomes ready
[ 36.060000] br-lan: received packet on bat0 with own address as source address
[ 46.080000] br-lan: received packet on bat0 with own address as source address
[ 56.100000] br-lan: received packet on bat0 with own address as source address
[ 66.120000] br-lan: received packet on bat0 with own address as source address
[...]
batctl s
tx: 12
tx_bytes: 1264
tx_dropped: 43
rx: 20
rx_bytes: 948
forward: 0
forward_bytes: 0
mgmt_tx: 365
mgmt_tx_bytes: 23182
mgmt_rx: 362
mgmt_rx_bytes: 22912
frag_tx: 0
frag_tx_bytes: 0
frag_rx: 0
frag_rx_bytes: 0
frag_fwd: 0
frag_fwd_bytes: 0
tt_request_tx: 1
tt_request_rx: 2
tt_response_tx: 2
tt_response_rx: 1
tt_roam_adv_tx: 0
tt_roam_adv_rx: 0
dat_get_tx: 0
dat_get_rx: 0
dat_put_tx: 0
dat_put_rx: 0
dat_cached_reply_tx: 0
and this is from NanoStation:
[...]
[ 17.460000] IPv6: ADDRCONF(NETDEV_UP): eth0: link is not ready
[ 17.460000] device eth0 entered promiscuous mode
[ 17.480000] IPv6: ADDRCONF(NETDEV_UP): br-lan: link is not ready
[ 17.520000] device br-lan entered promiscuous mode
[ 17.520000] IPv6: ADDRCONF(NETDEV_UP): anygw: link is not ready
[ 17.550000] eth0: link up (100Mbps/Full duplex)
[ 17.550000] br-lan: port 1(eth0) entered forwarding state
[ 17.560000] br-lan: port 1(eth0) entered forwarding state
[ 17.560000] IPv6: ADDRCONF(NETDEV_CHANGE): eth0: link becomes ready
[ 18.610000] IPv6: ADDRCONF(NETDEV_UP): eth1: link is not ready
[ 18.610000] device eth1 entered promiscuous mode
[ 18.620000] br-lan: port 2(eth1) entered forwarding state
[ 18.620000] br-lan: port 2(eth1) entered forwarding state
[ 18.630000] IPv6: ADDRCONF(NETDEV_CHANGE): br-lan: link becomes ready
[ 18.660000] br-lan: port 2(eth1) entered disabled state
[ 18.740000] IPv6: ADDRCONF(NETDEV_UP): eth1.11: link is not ready
[ 18.750000] IPv6: ADDRCONF(NETDEV_CHANGE): anygw: link becomes ready
[ 18.800000] IPv6: ADDRCONF(NETDEV_UP): eth1.12: link is not ready
[ 19.120000] batman_adv: bat0: Adding interface: eth0.11
[ 19.130000] batman_adv: bat0: The MTU of interface eth0.11 is too small (1496) to handle the transport of batman-adv packets. Packets going over this interface will be fragmented on layer2 which could impact the performance. Setting the MTU to 1532 would solve the problem.
[ 19.150000] batman_adv: bat0: Interface activated: eth0.11
[ 19.160000] 8021q: adding VLAN 0 to HW filter on device bat0
[ 19.210000] device bat0 entered promiscuous mode
[ 19.210000] br-lan: port 3(bat0) entered forwarding state
[ 19.220000] br-lan: port 3(bat0) entered forwarding state
[ 19.270000] batman_adv: bat0: Adding interface: eth1.11
[ 19.270000] batman_adv: bat0: The MTU of interface eth1.11 is too small (1496) to handle the transport of batman-adv packets. Packets going over this interface will be fragmented on layer2 which could impact the performance. Setting the MTU to 1532 would solve the problem.
[ 19.300000] batman_adv: bat0: Interface activated: eth1.11
[ 19.560000] br-lan: port 1(eth0) entered forwarding state
[ 20.810000] batman_adv: bat0: bridge_loop_avoidance: Changing from: disabled to: enabled
[ 20.810000] batman_adv: bat0: distributed_arp_table: Changing from: enabled to: disabled
[ 21.220000] br-lan: port 3(bat0) entered forwarding state
[ 22.650000] IPv6: ADDRCONF(NETDEV_UP): wlan0_ap: link is not ready
[ 22.680000] IPv6: ADDRCONF(NETDEV_UP): wlan0_ap.12: link is not ready
[ 27.590000] device wlan0_ap entered promiscuous mode
[ 27.620000] IPv6: ADDRCONF(NETDEV_UP): wlan0_ap.12: link is not ready
[ 27.630000] IPv6: ADDRCONF(NETDEV_UP): wlan0_ap: link is not ready
[ 27.650000] IPv6: ADDRCONF(NETDEV_CHANGE): wlan0_ap: link becomes ready
[ 27.650000] IPv6: ADDRCONF(NETDEV_CHANGE): wlan0_ap.12: link becomes ready
[ 27.670000] br-lan: port 4(wlan0_ap) entered forwarding state
[ 27.670000] br-lan: port 4(wlan0_ap) entered forwarding state
[ 29.140000] br-lan: received packet on bat0 with own address as source address
[ 29.670000] br-lan: port 4(wlan0_ap) entered forwarding state
[ 31.750000] IPv6: ADDRCONF(NETDEV_UP): wlan0_adhoc: link is not ready
[ 31.770000] wlan0_adhoc: Created IBSS using preconfigured BSSID ca:fe:00:c0:ff:ee
[ 31.780000] wlan0_adhoc: Creating new IBSS network, BSSID ca:fe:00:c0:ff:ee
[ 31.790000] IPv6: ADDRCONF(NETDEV_CHANGE): wlan0_adhoc: link becomes ready
[ 39.160000] br-lan: received packet on bat0 with own address as source address
[ 49.180000] br-lan: received packet on bat0 with own address as source address
[ 59.200000] br-lan: received packet on bat0 with own address as source address
[ 69.220000] br-lan: received packet on bat0 with own address as source address
[...]
batctl s
tx: 10
tx_bytes: 1096
tx_dropped: 41
rx: 18
rx_bytes: 756
forward: 0
forward_bytes: 0
mgmt_tx: 879
mgmt_tx_bytes: 51202
mgmt_rx: 352
mgmt_rx_bytes: 22292
frag_tx: 0
frag_tx_bytes: 0
frag_rx: 0
frag_rx_bytes: 0
frag_fwd: 0
frag_fwd_bytes: 0
tt_request_tx: 4
tt_request_rx: 1
tt_response_tx: 1
tt_response_rx: 2
tt_roam_adv_tx: 0
tt_roam_adv_rx: 0
dat_get_tx: 0
dat_get_rx: 0
dat_put_tx: 0
dat_put_rx: 0
dat_cached_reply_tx: 0
This problem isn't related to interaction between two devices, because also isolating a device via switch web interface, the other one after rebooting shows the same error again.
Again, I don't know if this is a libre-mesh related problem or a problem of batman loop avoidance.
Maybe the problem is related to this old closed bug? http://www.open-mesh.org/issues/170
Thanks,
Ilario
Hi all,
on the Libre-Mesh site I can read:
You need to install dependencies:
(i.e. in Debian 7.0 or higger)
apt-get install git build-essential libncurses5-dev zlib1g-dev gawk
after installing those packages and trying to compile I get
Checking 'non-root'... ok.
Build dependency: Please install the subversion client.
Prerequisite check failed. Use FORCE=1 to override.
make[1]: *** [tmp/.prereq-build] Error 1
make[1]: Leaving directory `/home/ilario/software/lime-build/build/trunk'
make: *** [checkout] Error 2
So subversion also is needed?
Bye,
Ilario
Hi!
On one node we are announcing a tunIn for internet and anotherone that have a
corresponding tunOut is tring to reach internet, but there is some problem,
packets never reach the other nodes and sniffing on interfaces never goes out
from the originating node, and on dmesg i have found some of this messages
[ 8829.860000] ip6_tunnel: bmxOut_9f4872: Path to destination invalid or
inactive!
That seems related ecause appears only on the node that is trying to go out,
is it a well known problem or it is necessary a deeper debug ?
