It’s rare that test and measurement is a glamorous role in building networks, and phrases like Multi-Dimensional Video Quality Management are enough to make your eyes cross, but the quality of video has become one of the core pinch points in IPTV deployments so we suppose that success here is at least glamorous.
This is the business proposition for 6 year old IneoQuest, and as such it becomes a ‘must have’ for IPTV so much so that it has mustered 300 customer networks around the world where it carries out video quality assurance.
Put very simply IneQuest monitors the condition of digital video, usually MPEG, throughout its journey from ingest right until it reaches a consumer’s TV screen and both isolates where problems are occurring, and tells an IPTV operator what to do about them
We talked this week to Stuart Newton, General Manager for EMEA, at IneoQuest, ‘Our IQ product family checks the integrity of IP encapsulated MPEG when it is first ingested at the head end, and then again at every stage in its journey to the consumer. When IP encapsulation has been carried out on it, from there it usually goes to a switch and we plug our appliances in right there.’
The nature of digital video is that a single packet is vital, and yet all IPTV systems lose packets. Telco networks were never designed to carry video data. Initially they were designed to carry voice, then data, now they have to carry data that has special vital packets, which make up the I-Frames which can be referred to across several seconds of video rendering.
What IPTV system designers tends to use to get over simple packet problems are either Forward Error Correction (FEC) systems or systems that conduct packet resends. They cannot use TCP for this because it would resend packets to every part of any given multicast, so packet resends are all proprietary and therefore not always reliable.
But another way to solve video quality is to make sure that a lot less packets are lost. The IneoQuest approach is to monitor the network and help tune it so that it is less susceptible to errors or more resilient against them
‘Previously the industry might use an MPEG analyser or a DSL tester or a network monitoring tool at different parts of the network to monitor traffic, but they were only testing one dimension of the network at a single point in time. The condition of the MPEG needs to be tested throughout the network.’ Hence that expression Multi-Dimensional Video Quality Management ‘ testing all the potential dimensions, all the time.
‘There’s a big distance between the video world and the world of IP networking. It’s very rare that you get anyone that understands the technology and language of both, and if they do they usually end up working for us.
‘If you have a UDP based multicasting network you have no way of knowing that a packet hasn’t arrived or has been damaged unless you monitor the network at every stage,’ Newton said.
But that’s a tricky problem to solve. The IneoQuest approach has been to devise tiers of different speed, deep packet inspection devices, which scale up 10 Gbps and down to DSL line speeds. They are built using reconfigurable parallelised FPGA technology. If MPEG data is damaged it can isolate at which stage in the network is was damaged, and then turns its attention to the tuning of the device which was managing the packet at the time.
‘In order to do video trouble shooting, you must catch it when it happens. A network has different loadings at different times and you won’t be able to reproduce a problem or work out what went wrong unless you have real time data. We keep all of our real time monitoring data for 10 days so when a customer calls in we can look up what happened.
‘What you used to have was what we call ‘voodoo’ trouble shooting, down to a kind of magic and guesswork,’ Newton then shows slide (below) where there is a problem near the head end, but because the first time anyone realizes that there is a problem is when a customer rings in, the operator sends out an engineer eating up OPEX. This has come about, he argues, because older point solutions for monitoring the MPEG condition at source, and the overall Network and the DSL last mile, all report no problem. But it turns out the MPEG was damaged way up into the network.
‘And what happens when you do send an engineer? In the old days he would swap out a set top, and that might fix the problem. If it didn’t you knew the answer was in the network, but how do you swap out an expensive DVR which has all the customer’s programming on it waiting to be watched? You can’t.’
‘And if it isn’t the set top, the video signal may have travelled over 1,000s of miles of cabling, and your engineer doesn’t have time to find the fault or a clue to know where to start looking.’
Once a device has been shown to cause a problem on a piece of video, there is the issue of what the network operator can do about it.
‘Every switch and router in the network needs to anticipate how much jitter it will get and be able to buffer it. One of the things we measure is if the delay factor going into a switch has changed by the time it has come out. If it has changed then the switch is using its buffer. We can set thresholds in these delay factors and then report any anomalies to a high level network management tool that can re-route traffic to avoid lost packets.
‘If it happens often you might consider changing one or other pieces of equipment or change its settings and then you simply get less packet losses.
And the problems are going to get worse not better as advances are made in IPTV services, says Newton, ‘Once IPTV services begin to have adverts inserted into video streams to support addressable advertising, there will be more and more places in the network where a video packet can be damaged. It’s going to make tracking video more complex and more important.’
The entire IneoQuest ethos is to stop finger pointing inside an operator, cut out any wasted OPEX resources, then it can point to a rapid repayment of the system. Now perhaps that’s a little glamorous after all.