routeros Archives - Networking HowTos

Fix Ethernet Port Flapping on MikroTik RB3011

October 13, 2020 MikroTik / RouterOS, Networking

The RB3011 router from MikroTik can sometimes have issues with Ethernet ports flapping, which can sometimes be traced back to using different link speeds on different ports on the one switch chip. The RB3011 contains two switch chips, both being the QCA8337 switch chip. The first switch chip being make up of ethernet ports 1-5, and the second being made up of ethernet ports 6-10.

Each switch chip has a 1 Gb/s connection to each of the two CPU cores (2Gb/s aggregated. See block diagram below). If you are experiencing port flapping, and you notice you have ports with different link speeds on the same switch chip ports (1-5, or 6-10), this article may help resolve that issue.

MikroTik RB3011 Block Diagram

source: https://mikrotik.com/product/RB3011UiAS-RM

The block diagram above shows the two switch chips, the ports linked to each switch chip, and how the switch chips link back to the main cpu cores.

Cause
The root cause of the issue appears to be caused by the switch buffer needing to be cleared before forwarding traffic to the other port, and when you mix link speeds, it cant always keep up, and flaps the port down/up.

Workaround / Prevention
The obvious simple workaround for this issue is to make sure all ethernet ports connected to the same switch chip are running at the same link speed. This may require turning off auto negotiation, and lowering the link speed manually. Ethernet ports 1-5 are connected to one of the switch chips, and ports 6-10 are connected to the other.

Resolution
This may not resolve the issue in all use cases, but it seems that the issue can be negated by disabling the CPU flow control option, which was introduced/fixed in RouterOS version v6.47 (stable) and v6.46.7 (long term).

Changelog from v6.47 (stable) and v6.46.7 (long term):

...
*) switch - correctly enable and disable CPU Flow Control on RB3011UiAS;
...

Unfortunately these options are not available from the WinBox GUI, and must be done using the CLI.

You can determine if you currently have flow control enabled or disabled by running the following command:

/interface ethernet switch export verbose

you should see something like this:

/interface ethernet switch
set 0 cpu-flow-control=yes mirror-source=none mirror-target=none name=switch1
set 1 cpu-flow-control=yes mirror-source=none mirror-target=none name=switch2

To disable the cpu-flow-control option, use the following command(s):

/interface ethernet switch set switch1 cpu-flow-control=no
/interface ethernet switch set switch2 cpu-flow-control=no

If you only want to disable it on one of the switch ports, only run the relevant command above for the switch chip you want to disable it for. See the block diagram for which ports are linked to which switch chip.

Mikrotik RouterOS Remote Vulnerability Exploiting the Winbox Service.

April 24, 2018 MikroTik / RouterOS, Networking

It was discovered on the 23rd of April 2018, that there was a remote vulnerability being exploited in the wild, that is exploiting the Winbox service on RouterOS based devices (Mikrotik / Routerboard devices). This is currently effecting RouterOS versions v6.29 through to v6.42 in the current channel (and up to v6.43rc3 in the release candidate channel).
Version 6.42.1 for current (and v6.43rc4 for release candidate), has just been released, which has fixed this vulnerability, and should be upgraded to as soon as possible.
While it currently remains uncertain exactly how the exploit works, it would appear that a remote user can connect to the WinBox port (which is port 8291 by default), and download a user database file, without successfully authenticating. The remote user can then log in, and take control of the router.
This will appear in the RouterOS logs, as a WinBox connection attempt that fails, and then a second attempt, which is successful.
This remote exploit relies on the WinBox service being accessible, so if you don’t have that enabled, or you are blocking it via a IP firewall rule, or are restricting users via the WinBox services ‘Available From’ list, you should be safe from this vulnerability effecting you. However, there is a chance that this could be exploited from inside the LAN by a malicious user if your rules allow access on the LAN side.
For more information on the exploit, please read the forum post on the Mikrotik site:
Advisory: Vulnerability exploiting the Winbox port
Here are some options to prevent your RouterOS device from being exploited. You may not need to do all of them to prevent this vulnerability, but the more locked down the router is, the better.

Block WinBox access

If you manage your router using another method, such as SSH, and you don’t actually need WinBox access, you can simply disable the WinBox service.

/ip service set winbox disabled=yes

Restrict WinBox access using the “Available From” list

Sometimes you may need to remotely manage a RouterOS device using WinBox. If the remote device that needs to connect has a static IP address that doesn’t change, you can set up a filter on the WinBox service itself, using an “Available From” list.
You can use the following command to set up a number of allowed IP addresses/networks that can connect to the WinBox service. Once this has been set, only addresses in the specified list of addresses will be able to connect.

/ip service set winbox address="192.168.0.1/24, 10.0.0.44/32"

In this example, the 192.168.0.* network is allowed, as well as the individual 10.0.0.44 IP address.
Keep in mind all IP addresses that you may need to add, to prevent you locking yourself out of the router, as this can prevent access from the LAN side as well.

Restrict WinBox access using the IP firewall

If you are using a version of RouterOS that has address-list functionality:
Create an address list called WinboxAccess and assign IP addresses and networks to the list

/ip firewall address-list add list=WinboxAccess address="192.168.0.0/24"
/ip firewall address-list add list=WinboxAccess address="10.0.0.44/32"

Add as many IP addresses/networks as needed to the address list.
Set up a firewall to block WinBox access from any addresses/networks that are not specified in the WinboxAccess address list.

/ip firewall filter add chain=input src-address-list=!WinboxAccess protocol=tcp dst-port=8291 action=drop place-before=0

This will add a filter rule at the very beginning on your firewall, blocking access to the WinBox service from any IP address not in the WinboxAccess address list created above.
In this example, the 192.168.0.* network is allowed, as well as the individual 10.0.0.44 IP address.
Keep in mind all IP addresses that you may need to add, to prevent you locking yourself out of the router, as this can prevent access from the LAN side as well.

Upgrade RouterOS to v6.42.1 or higher

Check your current version, and check for a new version of RouterOS.

/system package update check-for-updates

example output showing a new version available:

          channel: current
  current-version: 6.42
   latest-version: 6.42.1
           status: New version is available

If you are on a version between (and including) v6.29 through to v6.42, you should upgrade to prevent someone from exploiting this vulnerability.
The following command will download and install the latest update.

/system package update install

The router will download the updated RouterOS packages, install them, and proceed to automatically reboot.

          channel: current
  current-version: 6.42
   latest-version: 6.42.1
           status: Downloaded 23% (2.3MiB)

You can also upgrade the Firmware if needed.
Check the firmware version, and determine if you need to upgrade.

/system routerboard print

It should print out the version you are on, and if there is a upgraded version available.

       routerboard: yes
             model: 750G
     serial-number: xxxxxxxxxxxxxx
     firmware-type: ar7100
  factory-firmware: 2.27
  current-firmware: 6.42
  upgrade-firmware: 6.42.1

To upgrade the firmware, run the following command:

/system routerboard upgrade

You will be prompted as to if you want to upgrade the firmware. Press “y” to proceed.

Do you really want to upgrade firmware? [y/n]
y
23:47:59 echo: system,info,critical Firmware upgraded successfully, please reboot for changes to take effect!

Finally, reboot to bring the system up to date:

/system reboot

Press “y” to confirm the reboot.

Reboot, yes? [y/N]:
y
system will reboot shortly

Your router should now be up to date.

Change the WinBox port

While this wont prevent the attack, it may help in preventing probes, and future automated exploit attempts.

 /ip service set winbox port=8299

Please pick a random port number, and take note of it for later.
You will need to enter the port number, after the ip address, in the WinBox “Connect To” field, separated by a colon character.
eg.
192.168.88.1:8299
This option should still be done in conjunction with one of the IP filtering options above.
Please note that if you change the port number, and you are using IP firewall rules to limit access, make sure you update the rules to use the new port number.

Change your passwords

If you believe you may have been vulnerable to this attack, it is strongly recommended that you change your passwords on your device.
If the WinBox service was remotely accessible, its possible for the entire device to be compromised.

Further steps to lock down the device

While exploits like this are found from time to time, its best to keep your router as locked down as possible, to minimize the possible attack surface.
For more info on locking down a Mikrotik RouterOS based device, please have a read of my other article:
Secure MikroTik RouterOS Router via CLI

Secure MikroTik RouterOS Router via CLI

June 14, 2017 MikroTik / RouterOS

Mikrotik RouterOS devices are extremely powerful router devices. This can sometimes mean that the configuration of them isnt as simple as point and click for a new user. This howto will outline some recommended steps you can take to secure your Mikrotik RouterOS device, be it RouterBoard, a x86 install on bare metal, or a CHR (Cloud Hosted Router).
This guide will not go into detail on configuring the firewall. That is more complex and is going to be heavily dependent on your own scenario. As a general rule of thumb, block everything coming into your router, and poke holes in the firewall for the services you need.
Here are the steps to help secure your Mikrotik RouterOS router via the command line interface.
Change the admin password.
This first point should be the most obvious. The default admin account is blank, so the first thing you need to do is change the admin password.

/user set [find name=admin] password=Password123

Replace ‘admin’ with the name of the account you want to change the password for.
Change the admin username to be something more unique
In this example the default ‘admin’ username will be renamed to ‘test’. This can help prevent people guessing usernames and passwords and subsequently gaining access to your router. Its much harder to try and guess a password via brute force when you don’t know what the username is.

/user set [find name=admin] name=test

Replace ‘test’ with the name of the account you want to change it to.
Disable unused interfaces
Having interfaces enabled when your router is in a publicly accessable location can mean someone can physically go up to the router, and plug a cable in. To prevent this, disable any unused interfaces.
Show the interfaces on your device

/interface print

This will vary greatly depending on your hardware.
Example output:

Flags: D - dynamic, X - disabled, R - running, S - slave
 #     NAME                                TYPE       ACTUAL-MTU L2MTU
 0  R  ether1                              ether            1500
 1  R  ether2                              ether            1500
 2  R  ether3                              ether            1500
 3  R  ether4                              ether            1500
 4  R  ether5                              ether            1500

Disable one of the interfaces
In this example we will disable ether4.

/interface set [find name=ether4] disabled=yes

Replace ether4 with the name of the interface you want to disable.
Check to ensure its disabled

[admin@MikroTik] > /interface print
Flags: D - dynamic, X - disabled, R - running, S - slave
 #     NAME                                TYPE       ACTUAL-MTU L2MTU
 0  R  ether1                              ether            1500
 1  R  ether2                              ether            1500
 2  R  ether3                              ether            1500
 3  X  ether4                              ether            1500
 4  R  ether5                              ether            1500

The X in the 2nd column shows that it has been disabled.
Disable unused services
Show the services that are available on the router.

/ip service print

Example output

Flags: X - disabled, I - invalid
 #   NAME      PORT ADDRESS                                       CERTIFICATE
 0   telnet      23
 1   ftp         21
 2   www         80
 3   ssh         22
 4 XI www-ssl    443                                               none
 5   api       8728
 6   winbox    8291
 7   api-ssl   8729                                               none

Disable any unused services, especially the unsecure ones such as telnet.
This example command will disable the telnet service.

/ip service set [find name=telnet] disabled=yes

Note: Firewall rules should be set up to limit access to the services that you don’t want to disable.
Verify the service has been disabled

[admin@MikroTik] > /ip service print
Flags: X - disabled, I - invalid
 #   NAME      PORT ADDRESS                                       CERTIFICATE
 0 XI telnet      23
 1   ftp         21
 2   www         80
 3   ssh         22
 4 XI www-ssl    443                                               none
 5   api       8728
 6   winbox    8291
 7   api-ssl   8729                                               none

It would be best to disable all unsecured/unencrypted/unused services
Here is an example to only allow ssh:

/ip service set [find name=telnet] disabled=yes
/ip service set [find name=ftp] disabled=yes
/ip service set [find name=www] disabled=yes
/ip service set [find name=ssh] disabled=no
/ip service set [find name=www-ssl] disabled=yes
/ip service set [find name=api] disabled=yes
/ip service set [find name=winbox] disabled=yes
/ip service set [find name=api-ssl] disabled=yes

Disable MAC Telnet Access
The MAC server feature can be used to access the Mikrotik/RouterOS device by MAC address only, even if not IP address is set on the router. While it can be extremely handy on the trusted private side of the network, you shouldn’t have a need to have this enabled on a un-trusted interface.
View the state of the MAC server

/tool mac-server print

Example output

Flags: X - disabled, * - default
 #    INTERFACE
 0  * all

Disable the MAC server on all interfaces.

/tool mac-server set [find interface=all] disabled=yes

Ensure its disabled:

[admin@MikroTik] > /tool mac-server print
Flags: X - disabled, * - default
 #    INTERFACE
 0 X* all

Note: if you need MAC Telnet access, please configure it to only be on your trusted private network interfaces.
Disable MAC WinBox Access
This is essentially the process as above, but adding the mac-winbox option.
Check MAC WinBox server status

/tool mac-server mac-winbox print

example output

Flags: X - disabled, * - default
 #    INTERFACE
 0  * all

Disable MAC WinBox on all interfaces

/tool mac-server mac-winbox set [find interface=all] disabled=yes

Example output after disabling

[admin@MikroTik] > /tool mac-server mac-winbox print
Flags: X - disabled, * - default
 #    INTERFACE
 0 X* all

Note: if you need MAC WinBox access, please configure it to only be on your trusted private network interfaces.
Disable MAC Server Ping
Again, very similar to the above two features.
Show the status

/tool mac-server ping print

Example output

  enabled: yes

Disable MAC server ping

/tool mac-server ping set enabled=no

Verify its disabled

[admin@MikroTik] > /tool mac-server ping print
  enabled: no

Disable neighbor discovery on public interfaces
Neighbor device discovery can be a handy tool, but there is no need for it to be enabled on a public interface.
View current state of neighbor discovery interfaces

/ip neighbor discovery print

example output

Flags: X - disabled
 #   NAME
 0   ether1
 1   ether2
 2   ether3
 3   ether4
 4   ether5

Disable neighbor discovery on public interfaces
In this example, i will disable neighbor discovery on the ether5 interface

/ip neighbor discovery set [find name=ether5] discover=no

verify it is now disabled

[admin@MikroTik] > /ip neighbor discovery print
Flags: X - disabled
 #   NAME
 0   ether1
 1   ether2
 2   ether3
 3   ether4
 4 X ether5

Enable strong SSH Crypto
Check what SSH settings you are currently using

/ip ssh print

Example output

           forwarding-enabled: no
  always-allow-password-login: no
                strong-crypto: no
                host-key-size: 2048

Enable strong crypto

/ip ssh set strong-crypto=yes

Ensure its been applied

[admin@MikroTik] > /ip ssh print
           forwarding-enabled: no
  always-allow-password-login: no
                strong-crypto: yes
                host-key-size: 2048

Enable reverse path filtering
Check the IP settings

/ip settings print

Example output

              ip-forward: yes
          send-redirects: yes
     accept-source-route: no
        accept-redirects: no
        secure-redirects: yes
               rp-filter: no
          tcp-syncookies: no
    max-neighbor-entries: 8192
             arp-timeout: 30s
         icmp-rate-limit: 10
          icmp-rate-mask: 0x1818
             route-cache: yes
         allow-fast-path: yes
   ipv4-fast-path-active: yes
  ipv4-fast-path-packets: 0
    ipv4-fast-path-bytes: 0
   ipv4-fasttrack-active: no
  ipv4-fasttrack-packets: 0
    ipv4-fasttrack-bytes: 0

Turn on reverse path filtering

/ip settings set rp-filter=strict

Verify it is now enabled

[admin@MikroTik] > /ip settings print
              ip-forward: yes
          send-redirects: yes
     accept-source-route: no
        accept-redirects: no
        secure-redirects: yes
               rp-filter: strict
          tcp-syncookies: no
    max-neighbor-entries: 8192
             arp-timeout: 30s
         icmp-rate-limit: 10
          icmp-rate-mask: 0x1818
             route-cache: yes
         allow-fast-path: yes
   ipv4-fast-path-active: yes
  ipv4-fast-path-packets: 0
    ipv4-fast-path-bytes: 0
   ipv4-fasttrack-active: no
  ipv4-fasttrack-packets: 0
    ipv4-fasttrack-bytes: 0

Disable SOCKS server
The SOCKS server should always be disabled nowadays, unless you have a unusual use case scenario where something needs to use it. If this is the case, please ensure its not accessible from the public interfaces via a firewall filter.
Check the SOCKS server status

/ip socks print

Example output

                  enabled: no
                     port: 1080
  connection-idle-timeout: 2m
          max-connections: 200

In this example, it is already disabled (enabled=no)
Disable SOCKS server

/ip socks set enabled=no

Disable the bandwidth test server
Check the status of the bandwidth test server

/tool bandwidth-server print

Example output

                  enabled: yes
             authenticate: yes
  allocate-udp-ports-from: 2000
             max-sessions: 100

Disable the bandwidth server tool

/tool bandwidth-server set enabled=no

Confirm the service has now been disabled

[test@MikroTik] > /tool bandwidth-server print
                  enabled: no
             authenticate: yes
  allocate-udp-ports-from: 2000
             max-sessions: 100

Disable remote DNS queries
RouterOS has the ability to act as a caching DNS server. If you dont wish to use this feature, its best to disable it, if its not already disabled.
Check the status

/ip dns print

Example output

                      servers:
              dynamic-servers:
        allow-remote-requests: no
          max-udp-packet-size: 4096
         query-server-timeout: 2s
          query-total-timeout: 10s
       max-concurrent-queries: 100
  max-concurrent-tcp-sessions: 20
                   cache-size: 2048KiB
                cache-max-ttl: 1w
                   cache-used: 9KiB

Note: in this example, its already disabled (allow-remote-requests=no).
Disable remote DNS requests

/ip dns set allow-remote-requests=no

Other / General
Keep your router up to date. Bugs and exploits are found from time to time on all devices, and your router is no different. Please ensure you are running a current version of the RouterOS operating system.