SkyQ and Sonos interfering with each other

[woodchal]

This a reply by Oilmex in a post on the Sonos community from 4 years ago - I despair! 

I have 30 sonos items. about 15 wired and 3 boosts to serve the few sonosnet items.

My house has gigabit cat6 wiring and switches throughout.

I found after a wired sky Q installation(one main and 2 wired minis) that i got network flooding issues that caused even a pc to fail to connect to the wider internet my fast Zen fibre broadband and broke the box to box wired links between the 3 Sky Q boxes.

At present i have removed all but one sonos boost from the wired network and both sky q and sonos are happy enough (obviously the sonos big group performance is down a bit)

It is conceivable that a recently introduced Netgear GS108 V3 on my network randomly causes some kind of recursive flood. It does worry me that sonos hold a list of old switches that are 'not sonos compatible'

Sonos and Sky techies really need to talk to each other, as I figure Sky Q subscription is going to be highly likely for sonos users

If anybody asked I could not  recommend mixing SkyQ and Sonos

[woodchal]

@63johnw 

Didn't know that pictures were moderated - I have 2 dublicating posts - oh well

[Mad+Sweeney]

@woodchal 

Your old switches could have something to do with things as you point out.

There is one part of how Sky Q equipment connects over wired ethernet that I haven't been able to check.  It may be that it expects that wire links between the devices to be VLAN tagging capable.  I've sort of made this term up as I don't think there is an offical name for it but what it means is that any forwarding device such as a switch or media converter should take an entire ethernet frame and repeat it as is.

A problem with the way that VLANs work in ethernet 802.1q is that they effectively say that the type of traffic is VLAN in the place where it would usually say what protocol is used e.g IPv4 and then the type of the protocol really used  is bumped up behind it.  This makes the overall ethernet frame 4 bytes longer than the legacy expected maximum.

An old switch that doesn't know or care that 802.1q is a thing could do one of 3 things if an 802.1q frame arrives:

1) It just sees a larger than normal ethernet frame and just goes with it.  It works by accident.

2) It sees a larger than normal ethernet frame and considers that wrong so drops the frame.

3) It sees a larger than normal ethernet frame but presumes it would never be that big so leaves of the last 4 bytes when forwarding.

When traffic goes accross a link and it includes a VLAN tag, we say that link is a trunk, because there could be a mixture of frames for different VLANs going accrosss it. Non tagged traffic can go accross the link too, and a VLAN aware device that receives such traffic will adopt into the VLAN its configured for. Conversely traffic leaving and goes to a device that is not aware that the link is a trunk will have the VLAN tag removed.

If (and I don't know if this is the case), the Sky boxes treat the link as a trunk for [some] traffic between themselves (after all one Sky device can be programed to know what another Sky device is or isn't capable of) then there could be a mixture of tagged and untagged traffic going accross the link.

If your switches don't do 1) above then you'd be in trouble.

What you could do is remove the switches temporarilly and you may need to boot a few other things off the network in the process but just keep the relevent links connected by using RJ45 couplers which only cost a few quid.  If the cable runs are long then this may not work.  The other thing to try is to borrow some newer switches from somewhere.

One day curiosity will get the better of me and I'll enable port mirroring on a switch that sits between two Sky Q devices and see if any tagged VLAN traffic is going between those ports.  The reason I suspect this could be the case is from reading other threads where people have had things like powerline adapters between Sky Q boxes and all hell as broken loose.

Whilst most talk about managed switches messing up STP is all over the forum, I suspect that old unmanaged devices may be problematic too but for this other reason.  Nowadays anybody making a cheap switch or media converter will make it happy to forward 802.1q frames but in the past either it would not have been thought about, or somebody would have thought to themselves Why would anybody be setting up VLAN trunks in a domestic setting for which this product is marketed into? - Let's use the chips/firmeware we already have that does not handle 801.q frames correctly as that is cheaper .

The Wireshark trace looked fine BTW.

In summary, it is all a bit of a mine field and one can often not always know exactly what the equipment they own does, or is required to do yet wasn't relevant before when they introduce something new into the mix.  The thing is that more and more people will add more and more to their home networks and so there should be some kind of list of supported devices like unmanaged switches.  Most people doing that though will buy a new unit.

Of course there may be nothing wrong with your switches either and my theory about VLAN tagged traffic between Sky Q boxes over wired linksmay  be totally wrong. Only experimentation can tell because searching the net for the switch model and seeing if anybody has asked if they pass tagged VLAN traffic onward correctly might not yield an answer.   For some of the media convertors I use though that answer was available, so you could get lucky.

You'll figure this all out for sure, as whilst this stuff sounds complicated, it's not too tricky if you're able to find out what each device does or doesn't do and pair that up with what it should or shouldn't do.

[woodchal]

@Mad+Sweeney 

A small update - I have managed to get the Sonos system back onto to Wifi only (no devices cabled) and all devices are showing WM1.  So this takes the switches out of the equation.

The controller app can see and interact with all Sonos devices.  However the request for info from the TVBeam still does not return anything.  The browser just hangs.

[woodchal]

@Mad+Sweeney

If you are still following this then, I used wireshack to capture traffic to and from Play 1 and TVBeam.  There is a slight difference in the [] parameters for the return is this significant?

The outgoing traffic to …07 (Play1). Which responds

The outgoing traffic to …08 (TVBeam).  Which doesn’t respond.

The return traffic

[Mad+Sweeney]

@woodchal

So it looks like the reponse from .08 (TVBeam) does appear on the wire at it is the penultimate thing shown in the shot. If you were to click on it you'd see the XML that should appear in the browser.

There is a RST - Reset coming back to a different port on your machine. So two different ports were used to talk to .08.  It is hard to say, as the first one might have been the browser and the second one the Sonos app.

Given the Sonos app is now working to control everything, it looks like there is not much more to diagnose so it may not matter too much if pulling the device xml data from a browser works or not.

Since connecting via Wifi (not plugging stuff into the Sky equipmentt) is a more normal thing to do, I think you'll be more lucky with leaving it all this way.

I don't know if you know this but you can also pull up a network matrix by visiting any Sonos device on port 1400 at the path /support/review.  It's quite nice as each device is shown as both a row and a column and the squares where they meet show the connection between devices.  The grey squares show that 2 devices could talk to each other but this way but choose not to and the coloured in squares show where two device do talk to each other.  It's a nice way to see that Sonosnet is all good really.  In my setup you see that there is one device there that is able to talk directly to all other speakers apart from the rears/sub of the 5.1 setup as they connect via the playbar.  In this case this device is also the one with the ethernet jack and in the centre floor which makes sense for my house.

From a techy standpoint, now you've got Sonos on Sky's Wifi, any BPDU frames Sonos should reach the Sky devices without worring about your switches.  You're in the hands of Sky to do the right thing w.r.t Sonos working but this way they have more of a fighting chance with less unknown equipment in the way.  After all now the only thing between Sonos and Sky Q is air.

Am I correct in reading things. That for all intents and purposes, normal use of Sonos is now working fine accross your home and it was just that some manual diagnostic tests were not working out?

[woodchal]

@Mad+Sweeney Yes everything is working.  

However I suspect my OP issue would not be solved.  This is when I had the SkyQ wifi mesh operational and the app would only see the sonos devices that were on the same access point.  I don't really want to test this out as everything in the house can see the main Sky modem/router/hub.  However the garden (when the summer shows up) is not so good and one corner of the house can be a bit iffy.  

I don't understand what the problem is for Sonos to switch between devices connected to different access points but I shall just avoid for the time being rather than try to solve.

I did not know that you can also pull up a network matrix by visiting any Sonos device on port 1400 at the path /support/review - and even now I'm not sure to action this.

I will stick to the conclusion that Sky/Sonos should do more to resolve their differences.  

Any more thoughs on the sound drops, pixalation, glitches and general STP flooding that are all over the forum?

[woodchal]

Ok sussed out how to to the matrix thing - now to explore it a bit

[Mad+Sweeney]

@woodchal 

In the summer things may have changed, so switching the Wifi AP mode back on the TV boxes might not break anything, but for now I'd leave it all alone just to take break from it all 😁

I would not worry about things like STP flooding unless you are convinced it is affecting your installtion.

At a later date it may be worth swapping out those old network switches you mentioned with some others as a test when the oppurtunity arrises.  In all this, they were the only things that could possibly make your setup in any way unique. 

If you keep getting pixelation problems, I'd get an engineer around TBH.

[woodchal]

Thanks for all your assistance guidance and advice.

I agree I have probably reached my end game.  However, I did have a poke around the matrix etc you pointed me at.  I have watched a very useful guide to this on Youtube and must say didn’t see what I expected. 

So, my devices are indeed listed down the left column as expected, but each device seems to have two entries with adjacent MAC addresses, across the top of the matrix.  Crafting is ….38 but there is a….39 which seems to have the information regarding signal strength, with no indication of the route being taken (as per the info on youtube). Do you know why?  I suspect that this is because the system is on the Wifi rather than the Sonosnet and if I plugged a cable in, I would get a different result (tonight’s trial). 

On all of devices the /usr/sbin/brctl showstp br0 call returns that STP is disabled for this interface - this is a puzzle to me

[Mad+Sweeney]

@woodchal 

The picture is it not moderated yet but I can answer that last question with some preamble.

So all Sonos speakers have at least one wired port and one radio.  These are named

ethX (eth for ethernet) and athX (ath for Atheros, the wifi chip maker).  A virtual bridging device br0 is then created over these.  The ports have their own MAC addresses but the bridge presumes the MAC address of the ethernet port.   Devices with 2 radios and therefore an ath0 and ath1 are those which have an additional 5Gz radio for surround sound support.  Such a device which is not actaully setup for surrond will not show its 5Gz radio and I presume it just switched off in that case and will just show ath0.

So having 2 Mac addresses is normal.  Usually the one for the wired ethernet is what is printed on a label.  The same applies to the Sky Q kit in this regard.

So the reason STP would be disabled I presume is because by connecting over Wifi.  The sonos radios all try to talk to your access point.  Presuming that they all can do so and  if you drew this out as a diragram you can see there is no way this could (or should be able to) create loops, so STP would be a waste of time.  If you had more than one AP with the same SSID then loops would again become possible if some devices were on one AP whilst the others were on another (sound familar?)

As soon as you plug a cable in, the Sonos will thing to itself, well that cable could be plugged in anywhere including media that bridges to devices already accessable via radio.  It would likely start running STP if you did that.

I did some other reading and I think Sonos is super smart and if for example you put a speaker out of reach of the Wifi access point, it would give up on talking to your access point and try to find a nearby Sonos speaker (or Boost) instead.  In this case it would be running in a sort of hybrid mode.  It might switch STP back on in this case too if perhaps it can flip between the sonos net and wifi if both signals are present but Sonosnet looks better.

The more I think about it I do think your old switches could have been your problem when you were in a multi wireless AP scenario.  If the BPDU (STP) packets from Sonos were being lost between Sky devices due to your switches then it would not be able to prevent these loops.  Or perhaps it would never have worked regardless.  I still think it's worth waiting a while before doing any changes though.  Now just learning everything is more usefull, as usually when comming back to a problem a long while later and trying to solve it better, the longer you wait the better.  I often have lightbulb ideas about stuff months after bodging something.

[Rhonny]

@Mad+Sweeney @woodchal

Why don't you two PM each other?! I think this thread is now of little use to anyone. No-one is going to have enough time alive on this planet to read the dissertations you've both written between you! 😉 

[woodchal]

Rhonny

Thank for sharing that and I do understand what you are saying.

However, for me, just let me say how refreshing it has been to find somebody on this forum, who is clearly knowledgeable and happy to share their knowledge.

So many regular contributors responding to new forum members, who have ended up on forums like this just seem to give curt answers that in effect say “look at me I’m clever” or “my set up is fantastic” or “bad workman blame their tools” or “it’s your fault” or endlessly requesting more information.   Let me say this is not directed at you personally, just a general observation.

I have learnt a lot about how to diagnose Sonos v SkyQ and this post will sit there as a resource for others who come looking for solutions to resolve their issue in the future.  I would have loved to have fallen over this post when searching for solutions or indeed just to understand the technology behind my issues.

The key takeaway for me is SkyQ + Sonos is likely to be a common set-up.  These companies need to get their act together with some interoperability tests for new releases.  Both companies seem to be quite “technically arrogant” relying on past reputations and they need to restore putting consumer experience at the heart of their operations.

[Dunshavin]

Huge thanks to @woodchal and @Mad+Sweeney for putting all their time into this. I have this problem, and everything I've read here just shows me that it needs resolving by Sky and Sonos. There is nothing that your average user can do to fix this.

I hope that the sheer quantity of good research and insight here will be noticed by representatives from both companies, and they will assign some engineering to time to it. I guess they have to be very careful that they don't 'fix' something here, and break it for another set of customers.

If anyone has a eureka moment (you know, 'just flick this switch here and will all be fixed') please do post again to this thread!!

[Mad+Sweeney · Accepted solution]

I've proven some of the stuff I suspected was happening here actually is - tl;dr -

• Scenario is quite obscure - depends on topology.
• Sonos can drop data but Sky Q is passing data that standards would dictate that Sonos should drop.
• Other 100Mbps equipment like Sonos might do the same thing.
• Have not tested what happens if Sonos is made to bridge traffic between Sky Q boxes wirelessy. This might happen by accident in a mesh.

-- more detail --

It is a reasonably obscure scenario to get into either by wriring up a particular topology in your home, or perhaps if certain links are made to be up or down in the overall mesh of the two systems.

The problem occurs if a piece of Sonos equipment (perhaps any 100Mbps equipment) is used as a bridge to forward ethernet frames that use Sky's tunnelling protocol and that these frames are too big for Sonos to forward.  What is in these frames un-tunnelled can ironicaly be the Sonos traffic that could have previously passed through that Sonos device before in the opposite direction.  My broken topology did put this constraint on a Sonos device (a Playbar in this case).

So the Sonos is at fault? No.

In this case the Sky Main box is sending a frame to the Sky Hub (I think the laptop up there moved onto the Hub's Wifi AP different from my pictre above) that has a size of 1528.

This was gotten from Wireshark monitoring a mirrored switch port and setting the network interface in Mac OS to allow jumbo frames.  Otherwise my computer itself would sillently have dropped these frames and I could not have shown them here 😃. Also to show the tunnelling you must add this configuration to Wireshark.

decode_as_entry: ethertype,29812,(none),Ethernet
decode_as_entry: ethertype,29920,(none),Ethernet

The maximum size that should appear to be more compliant is 1514.  That is the MTU of 1500 and the 14 bytes that make up 2 x 6 byte MAC addresses and the 2 byte protocol.

So the Sky tunneling protocol is broken as it can not tunnel through devices that limit their frame size in a more standard way, and may be limited from allowing anything more by their hardware too.

If the inner frame were 14 bytes or more smaller then it would make it through, so pings and smaller TCP streams will make it through.

A a fix in this lab test like topology would be for the MTU as seen by the OS inside Sonos to be lowered by 14 bytes.  This would make the tunnelled Sky Q frames smaller because the original Sonos frames before tunnelling would be smaller.  It would only help Sonos traffic and not other traffic that got tunnelled.  It would be a silly thing for Sonos to do really anyway.

What Sky will have done with their protocol is have been sneaky and tried to take advantage of the fact that newer equipment supports larger ethernet frame sizes in order to  support things linke 802.1ax which gives an extra 22 bytes to play with, though these are supposed to be in the headers and not in the protocol data unit. 

@woodchal  Are your old switches that were/are sitting between Sky equipment only 100Mbps by any chance?  If they are then swap them for Gigabit ones.