July 12, 2011 | Jim Killock

Open Internet: how to be open about how closed you are

Today, the Broadband Stakeholder Group had the second meeting discussing what to do to protect the Open Internet: a process started after Ed vaizey’s meeting including Sir Tim Berners-Leee.

Today was what should have been the easy part: talking about transparency of ISPs over network discrimination, or “traffic management”. You can see that all major ISPs have now published a standardized policy such as this from Sky or this (sigh) jpg from Virgin

However, it emerged in discussions that:

  1. The policies are not especially transparent without understanding the real policies, especially for developers. What for instance is "P2P" including or not including? What is “game” traffic?
  2. They are published in fairly poor and varied manners currently, without explanation as to how they may be reused, if there are copyright issues; open formats and licenses could help here
  3. There were a number of appeals for the "real" or raw information sao developers and tech-savvy customers could understand what is really happening, and indeed verify it,
  4. There were questions around verification, including independent verification, and customer tools to verify the information

The substantive question: whether transparency would work or not was not really discussed. But it is at least useful, especially if real transparency is created. If you have suggestions as to what data you would like to see, or how customers should analyse and verify the data, please let us know.

Comments (10)

  1. Chris Watts:
    Jul 12, 2011 at 06:02 PM

    It would be good to know about port blocking, particularly for mobile networks. Are the mobile providers included in this discussion, or is it only fixed broadband?
    Secure links can be disrupted by interfering with certificate access, anyone doing this?

    1. ad3y:
      Jul 12, 2011 at 08:22 PM

      It would be good if ISP's listed exactly which sites (DNS and IP Addresses), protocols and ports they blocked and the reason why (including a link to further information if available).
      They should make this list obvious (a link to the relevant page) on their Support homepage so that anyone could check it and ensure they were not blocked from legitimate websites/protocols etc.
      This list would have to include everything they block, no hiding or secrets!!!

      This would ensure that end users could compare ISPs based on the blocking they do as well as trust ISPs not to be a tool of oppression.

  2. Jim Killock:
    Jul 12, 2011 at 08:18 PM

    Mobile providers are involved, yes. I’m not sure if the legal obligations are different, but we can certainly ask about this.

  3. ad3y:
    Jul 12, 2011 at 09:12 PM

    Also, if ISPs are logging what you do when online, you should be able to easily view the logs for your own connection to see what data they are recording about you.

  4. Graham:
    Jul 13, 2011 at 10:02 AM

    The full technical description is complicated: many DPI vendors compete on the basis of proprietary and complex algorithms to detect different types of traffic (which may go beyond simply looking into the packets but may also include, for example, analysis of the flow of packets, sizes of packets, etc). In those cases, it will be impossible to specify the exact algorithm (the ISP won't even know, and they will get improvements in each software release). Even if they did know, they would not publish because, in those cases, they do not want the software they are trying to block to know what changes to make to bypass the blocking -- it is an arms race.

    However, in those cases there is a very clear goal: to block or manage some particular sort of traffic (for example, Skype). They should be explicit about that. "P2P" is not acceptable, nor is "VoIP" -- but "bittorrent" and "Skype" probably are. We (the consumers, and comparison websites, helped by the regulators) need to make sure that there is an active market so it is in the commercial interests of the ISPs to be as precise as they can: if they are actually blocking Skype and they say VoIP then all users of VoIP will refuse to use them.

    What is important is that if there are particular types of destinations (countries, domains, etc), or particular IP ports which are managed, blocked, or interfered with (e.g. transparent proxies) then these need to be listed with enough information to allow a technical expert to be able to understand. For example, if the policy is that traffic in the UK is shaped one way, traffic to the rest of the world another and traffic to akamai a third way, that is an adequate way to describe the destinations -- actual IP address lists are not necessary. However, it is not adequate to state that email is transparently proxied -- does that mean just smtp on port 25? what about smtps or TLS? other ports? Enough detail must be provided to allow a technical user (for example at a comparison website) to understand the performance and security implications.

    And, of course, the actual shaping must be described -- that is the most critical part! Is it just a monthly byte limit? or a bandwidth limit at peak times? or blocking? or proxying? or something more complex. In this case, I see no reasons why we cannot insist that the actual algorithms are published (the ISPs won't want to because this is actually competitively important information about how to make the most efficient use of your network investment).

    1. Graham:
      Jul 13, 2011 at 10:15 AM

      By the way, I must declare an interest. In my day job, I do marketing for products which include policy control products which would be involved in this traffic shaping (although no UK companies are using these products). Personally I am a strong believer in being completely open with your customers, and I do not believe that what I am saying favours our products over our competitors in any way. However, you need to know that I do have a commercial involvement in this business.

  5. Ben Weiner:
    Jul 13, 2011 at 10:09 AM

    Any chance we could make a standard template that groups the techniques together and give that to the ISPs to fill in (or help them to fill it in)? It coud be a tabular list giving each technique with tick/cross/question marks for the big picture and then a linked long-form text answer for those who really want to know what the ISP says it is doing. Helpful?

    1. Jim Killock:
      Jul 13, 2011 at 03:38 PM

      Absolutely, at least two layers are needed. We have the chance to get ISPs to agree to provide the information, so we need to specify that as much as possible.

  6. Charles:
    Jul 14, 2011 at 04:02 PM

    Virgin Media does have a proper policy page, with the table as a table with text in. But, Virgin Media keeps moving information around its website and putting it in obscure places.


    As Graham asked, what is peer to peer? Virgin Media’s policy has very subtle wording. Both Skype and Spotify have a peer-to-peer component, but don’t rely on every client being able to use this. The policy states that the basic functionality of both applications will work, but not that none of their traffic is throttled. So, they might not work as well as they could.

    For static traffic shaping, we want the IP addresses or domain names, the IP protocols and the port numbers. This would be for white as well as blacklisting. For techniques that use deep packet inspection or that use packet size and timing statistics, we want to know what applications or application-level protocols are being targeted. If only some part of an application or application-level protocol is being targeted, which part. And, of course, numerical values for the relative bandwidth restrictions and the times when they are applied.

    I don’t see the arms-race argument being a valid one. Someone producing a popular application for use all around the world isn’t going to make major changes to their software based on what traffic shaping is being performed by one ISP in one country this week. You do see it with anti-censorship software, but those circumstances are significantly different.

    1. Charles:
      Jul 14, 2011 at 04:14 PM

      Jim, your webmaster needs to do something about the website’s comment submission. My first comment took me quite a long time to post. If you don’t enter an email address, and the form doesn’t say it’s mandatory, you get the following error message.

          The reCAPTCHA wasn't entered correctly. Go back and try it again.