Does Europe fracturing Regarding respect for privacy? -

Last decade, all of Europe was walking hand in hand towards nightmare monitoring companies. This decade, that has changed, and some countries in Europe (Germany et al) choose to restore civil rights while others (UK et al), press to Big Brother and beyond. This is related to a showdown over a few years.

For a time after September 11, 01, everything was crazy. Two years before, a high-profile report of a domestic spying scandal in European countries concluded that "even a small note scribbled in the margins of a personal file, a note that is never used for anything, is always an invasion of privacy. "After 01, the floodgates opened and there was no end to the ways that have been justified by the ends.

If a politician had suggested to government tracking devices to all citizens in 1999, their career would be over in ten seconds. After 01, it became a real thing, known as the Directive on data retention. It was approved by the European Parliament December 14, 04, and was effective until April 8, 2014, when the European Court of justice not only said that the directive on data retention was more indeed, but it was so serious, he never in force. The Court modified the laws and fundamental rights restored retroactively.

For several years between 01 and 08 approximately, it seemed that something was. There was no invasion if it just crazy enough "to keep people safe."

However, this government monitoring of citizens was challenged in various European states, even before his arrival at the European Court of Justice (ECJ), the European equivalent of the Supreme Court. The ECJ came to the verdict on April 8 this year that the directive on data retention is completely invalid if it ceased to exist as a whole, not just in part.

In early 2010, when the data-retentionist the type of monitoring should be implemented by all countries in Europe for several years, nine states of Europe 27 (now 28) were either still refuse or had implemented and had had a national Supreme Court strikes down. A full third did not meet.

The problem with data retention as such is that it treats everyone like criminals. Specifically, it invades privacy before any individual suspected of a crime has occurred, to everyone , if such a suspicion should appear By . The criticism has not affected the ability of application of the law to track people suspected of crimes. The criticism concerned the enforcement capacity of the law to follow the people who are not suspected of crimes. There is a fundamental difference, and the European Court of Justice highlighted this point :. That the very basis of core data retention, data collection bulk without distinction, is unacceptable

This strong states of Europe from where we see pulling in different directions.

Around 06, it was just a race to the bottom. Not anymore. Germany was one of the first states in Europe to challenge the retention of data - more specifically described as "Total Communications Logging combined with physical Tracking Too" - at the Supreme Court, and court struck down with a hammer of Fury, thanks largely to the excellent footwork of a militant group known as AK Vorrat (short for about "working group on maintaining" ). following the decision of the ECJ, the ministers in Germany have confirmed that data retention is now stone dead as a concept. other countries, such as Austria, follow the same line

countries

At the other end of the spectrum are like UK , described as an "endemic surveillance." - the lowest ranking possible - by Privacy International on time. today, there is a legislative initiative to restore all of this logging citizen monitoring and communications whatever verdict in the Court of justice, but do it everywhere as an initiative national. Basically, while ignoring the ECJ said, and also claim that it is a good idea to treat all your citizens as suspects at all times. Other countries, such as Sweden and Denmark, are following this line too.

Last decade, all states in Europe walked hand in hand to abolish fundamental rights. Suddenly, there is a clear divide between European states will come to a showdown on fundamental rights as such. In such a showdown, there will not be anywhere to hide for politicians who have pushed for a Big Brother society, which is basically good - even if slow -. Development

It's always good when there are different voices saying different things. It is then and only then that bad ideas, ugly and serious can be seen for what they are.

Privacy remains your own responsibility.

News leaks Privacy Highlights Catastrophe How Nus are nothing -

A group Criminal obtained personal details for one of the largest anonymous forums Sweden. Made to frighten, intimidate, and political opponents in the forum under the protection of anonymity, collateral damage is immeasurable.

People have never been fired for private photos exposed. However, people were fired, expelled, and divorced for private opinion exposed - and still are. This highlights the asymmetry privacy leaks are reported on widely (nude) and those that are not (violations of anonymity). It is disheartening to see the high concentration of oldmedia on anything sexually distance, combined with the total lack of interest in things that build (or threaten) the foundations of society.

Anonymity is such a foundation. The ability to speak anonymously is a key principle of freedom of expression, unless you're in the Cuban version:

"Of course, you have the freedom of speech, but I can not guarantee freedom after speech. "- Fidel Castro

opinions should be able to exist without considering someone for them, or the company will not evolve

Everyone naked from time to time .. the children, as evidenced by, well, children. While it is supposed to be kept secret, nobody is really surprised or shocked that this happens then when the time surface images in time (like behaviors have evolved to take pictures of everything today), it's embarrassing - but not much more

When bad surface of opinions, it ends the career of the roof and somebody marriage. What's worse - this is sometimes the result fine of those who seek to identify someone behind a policy, but anonymous opinion. But the anonymous opinions are as crucial for democracy that secret elections.

Freedom of expression can be summarized as "The right to utter cretins to be completely false" and accurate. This is a good thing

freedom of expression has never been necessary to protect traditional :. Expressing opinions that kittens are cute and Game of Thrones takes too much time between seasons. and when someone says something well respected despicable, they are still protected by the institution. However, when someone scorned said something totally unpleasant, is where freedom of speech intervenes to protect the declaration.

this is what freedom of expression is to

for in the end, it can turn in one of two ways. either despised were wrong, in this case, not hurt. Or they were right, in this case, they did a great service to civilization by taking the risk of exposing with a controversial statement.

Examples of such statements that may just until recently ended someone's career, marriage and housing:

• "people arriving to be born homosexual must have exactly the same rights as all others. "
• " Maybe we would all benefit from the legalization of cannabis. It seems that the alcohol industry and the prison are those who pay more to keep illegal. I wonder if there is a connection. "
• " I think people who come to be born a woman should be able to have a career like everything in today's workforce. "

Much less that there is a human life, it is extremely controversial statements. They would get you shunned . It is thanks to troublemakers who exercised freedom of expression, sometimes a necessary cover of anonymity, that the company has progressed. (Unfortunately, in many parts of the world, these statements are still controversial.)

This even begin to go into collateral damage opponents outing for political scare into not exercising their freedoms of expression and opinion:

Imagine if each disposable account on Reddit was suddenly doxed and identified . just / r / offmychest would be a disaster. Not to mention / r / abusevictims. This is the local equivalent in Sweden, and it just happened.

This criminal group caused damage incommensurable to the fabric of trust in society.

Privacy remains your own responsibility.

FBI has no morals Leg To Stand On Criticizing Google, Apple Over Encryption Phone -

the FBI keeps hammering at Google and Apple, saying that encryption by default cell phones allows all types of criminals. Frankly, it's disgusting. The FBI, which is. They do not have a single legal leg to stand on. Imagine if construction builders had started keeping the master key to all buildings they built, all private residences, even after these homes had been sold and that someone had moved there. The manufacturers would not do it out of spite or malice, but just as a result of not really thinking about what they were doing. Imagine if things sort of just developed that way in the building industry, and nobody reflects much about it. Then the law enforcement discovers that there are key mansions somewhere, they just can not pick them, so they go to these entrepreneurs to require random key private homes under the threat of strength. At this time, the application of the law would be able to walk in and out of private homes without warrant, simply because entrepreneurs conveniently kept a master key. You can see that happening, can not you? Just as law enforcement entered hospitals and blood samples taken from suspects for DNA matching. (Yes, they did.) At this stage, entrepreneurs are finding that the backup master key to private homes sold was not a very good idea, and decide to stop doing something. This is where Google and Apple are regarding mobile phones today. So furious Strawman objections FBI to Google and Apple essentially amount to "If you do not keep the master key for the super-private devices you sell, we can not force her to help us to walk straight into the super private aircraft at leisure and read everything that we love. " Yes, this objection is true. But this is not an undesirable side effect. It is the whole point first crazy. it does not matter if encrypting your personal data "using kidnappers," the FBI claims to desperately. it is doubtful, but it is also completely irrelevant. it does not matter if it helps horrible person. Application of just law is not get to your house master key to walk at leisure, period . They need a search warrant. and even with a mandate , you are under no obligation to open the door for them; they have a legal right to try to break down your door, but no right to fictitious succeed in the break. If they can not bust down the door (or open safes), even when it is legal to do so, no luck. This is not your problem, it should not be. The objections of the FBI is deeply immoral, and they must not only be called on it, but called for how they behave appallingly. They must be publicly and visibly torn down their large imaginary horses. Privacy remains your own responsibility.

At least 26 people who had nothing to hide Tortured by CIA -

twenty-six innocent people have been tortured by the CIA. These were people who had literally nothing to hide, but they had something to fear anyway. Civil liberties are to be applied to everyone without exception or will be reliable for anyone.

So the nuke this week was the torture report, showing that the CIA had used indiscriminate torture leading to the death of detainees. The CIA was literally more inspired by 24 television series by the laws and constitutions. In several cases, torture has resulted in death -. What we would normally call "murder"

According to the British lawyer and writer Philippe Sands, Jack Bauer - played by Kiefer Sutherland - was an inspiration at the beginning of "wagging -méninges "meeting military officials at Guantanamo in September 02.

This is not only a violation of a principle. It is a serious violation of roughly all of them to together : Ban unusual and cruel punishment, presumption of innocence, the right to a speedy trial the right to life, prohibition of collective punishment, the list continues.

According to the report, the CIA knew that they had the wrong person completely in at least 26 cases.

26 people who had literally nothing to hide.

This shows a very brutal way that it does not matter if you think you're white as snow and have nothing to hide it is. The saying "nothing to hide, nothing to fear" is false, dishonest and misleading. It is always, always, always someone determines whether you have something to hide or not. And they are never using the same criteria as you are.

Here, 26 people who believe in "nothing to hide, nothing to fear" were rewarded by torture. A decade long torture even. Does that mean that if you protect your privacy, you may torture? No, of course it does not.

But what not means is that if you buy into the rhetoric of "rights should not be considered for people who have something to hide," you buy in the idea of ​​civil liberties being flexible and useless - in the illusion that they still apply in some way to anything you when the time came, thinking that you have not done bad. of course, there are no such guarantees, and if history is any guide, they not . civil liberties are either held for people who least deserve it, or person at all. There is no gray area about it.

Privacy remains your own responsibility.

Link: Understanding electronic combination Tracks You Leave Behind -

In the film Citizenfour , Jake Appelbaum is seen briefly give a security primer for activists. He speaks link: what happens when you provide two pieces of identification at the same time, and how it means they are always linked. The identity should not be an identity card; it could be a metro card or a mobile phone

The danger to privacy does not primarily lies in when you log in using a method -. MAC (unique address of your network card), IP address, credit card, login, IMEI (unique identity of your phone), et cetera. The danger when two of them are interconnected.

This is not the tracks you leave behind. It is the combination of different tracks you leave behind, and the intersections of these tracks.

If ever you're using your laptop on a Wi-Fi network, the network knows when it's back to you, for example, because the wi-fi component on your laptop comes with a unique name. This unique network component name (called a MAC address) can be shared across networks, and only one of them need to know something more about you, for each of them to know about you. Did you ever use a credit card somewhere to get access to a public wi-fi, for example? Enough to every public wi-fi to know you by name -. Not only move forward, but historically as well

This is why it is crucial not to link the accounts together. Do not use your primary email when signing for a anoymization service. Do not use your regular credit card or a credit card with your name at all, or a first credit card when signing with an anonymization service. (There are four, I wrote that you should never trust a VPN service that accepts bitcoin. PIA accept Bitcoin, of course.)

More importantly, the cross has not to be automatic, nor to occur simultaneously.

Let me take a concrete example of the Swedish Pirate community to illustrate this in practice. It was five years ago, there was a new person in the comment field that claimed to be 20 something woman pirate sympathizer, but that was quite aggressive and inflexible to change things radically. Overall, this person acted rather abrasive and demoralizing in the comment field on a number of blogs

We did not know who it was, but we had a feeling that something was not quite right -. That person was not who they claim to be. So a group of us used the only thing we had - the IP address of the comments - and just kicking the WordPress search field on two dozen different blogs, each on their own, much greater scope than that in which he had been demoralize the community.

Bingo. The same IP was used almost a year before on a totally different blog for two random snarky comments. Now, that would not normally be enough to claim was the same person. - If there was not the same style and language that this person was actually a 50-something with the copyright industry

This is what we have do using nothing but a stock WordPress install, no web server logs, nothing.

(How we use this information? One of us asked in an answer to a sarcastic comment, "Ohai again, Y. you use the same computer that a person named X used ago one year. you two know each other? "the so-called 20-something woman was never seen again after that.)

now, it was a concrete example of something that is not built monitoring at all, simply using two different sites in a dozen nonprofit community. Then consider for a moment what the IMSI sensors, and TimeStamp boxes that record the identity of each mobile phone from within a range of 500 meters, are able to do when they are deployed in tens of thousands. (Changing your phone is not enough - if you repeat the same pattern that you have done with your old phone, all unit as the travel schedule or society or similar, you are very likely re-identified .)

Privacy remains your own responsibility.

Have NSA wiretapping Bulk Since 1976. They will not care about what happens to the Patriot Act of 01 - .

right now, there is a debate on a small section of the Patriot Act in the US, and best option removes the authorization of the US NSA wiretapping in the world. Both answers in the debate are wrong. No change in the law will stop the behavior of the NSA: they were eavesdropping like this since at least 1976 and do not care about changes to a law of 01. It is to be the most convenient justification of the day. If this justification is removed, there will be countless others.

The entire debate is a red herring. Getting rid renew or Article 215 of the Patriot Act, rewriting with a more limited scope, if not get rid of the Patriot Act in its entirety, absolutely nothing to change the behavior of the NSA.

The NSA is simply choosing to justify his wiretapping conversations and bulk metadata collection with the Patriot Act currently. If this particular avenue is closed, there will be another justification. And another. And another. And at the end of the day, they will both know and do not care that they break all.

Let me tell you about an event in 08, when I was on the same expert panel that the local supervisor of the NSA activities. (It was not his official title, of course. He was director of the Swedish FRA, which is the Swedish equivalent of the US NSA, but the former has acted like a local accomplice to this latest). In all cases, it is difficult to get much earlier in the intelligence community.

When lunch arrived at the seminar of the event, that person joined me at my table. We were the only technical geeks out there, it makes sense in a weird sort of way. A proposed Swedish equivalent of the Patriot Act, but on steroids, has been discussed everywhere at the time. (Among other things, it expressly authorized the NSA to analyze Swedish and maintain a database on sexual orientation of individuals, to give you an idea of ​​how far it went - no, okay.)

I told him that many people at the time believed that the FRA had already bugging all loose son.

"Oh no," he said. "We do not do it. We're listening [everything sent over] satellites. We have done since 1976. This violates both the Swedish Constitution and the European Convention on Human Rights," he sneered.

I got the impression that he was actually boasting, might try to recruit me, maybe try to find technical hand. But at the time, he did not know that I record the conversation.

I sent the recording to the press after. So what does that happen? Was there an outrage? A storm? Is it provokes political pressure to cease violations?

No. The press does not mention at all, with one exception - an article in Computer Sweden, a technical magazine for IT professionals. There was not a single mention in traditional oldmedia. Not one.

There was a lot of blogs wrote about it (in Swedish), though.

The official justification for wiretapping all calls that have been transmitted by the satellite link is an obscure radio law, who said that no one can be limited to pick up radio waves air. However, this law was intended to cover radio broadcasts, and certainly not the construction of a huge satellite dish twice alongside the recipient of satellite dish, and listening to a copy of every sent even if the transmission is be found radio waves. Lawyers have just given an empty Nope official excuse for this, talk about expectations of privacy, as the FRA overdirector said he knew to be the case in a recorded conversation.

Notice of this episode that although is an official external legal justification, the top brass know very well that what they are doing is completely illegal at all levels. More importantly, you will also notice that they do not care a bit it is illegal for the simple reason that they should not care.

It boils down to this :. When the problem is that the NSA and their accomplices agencies do not care a bit what the law says, the solution can not be to change what the law says

At the end of the day, there are exactly two ways to stop the bulk wiretap NSA correspondence: ..

you can cut their funding

Or you can cut their electricity

any other tweak to their environment, including their legal, has no effect.

Supreme European Court: Due to the NSA, US companies are not self-agency to accept Privacy Obligations -

This week, the European Court of justice - the highest court of the European Union - said that American companies can not transmit sensitive private personal data out of Europe to United States for treatment because they have so far. It was a cancellation of the so-called Safe Harbor, where the self-declared US companies that meet certain European privacy standards. But the European Court (ECJ) the reason of Justice to declare Safe Harbor void goes well beyond the cancellation as such - it is said that US companies do not have to Agency such promises of any kind in the first place, contractual or unilateral, not now, not ever, as long as the NSA works.

most have focused on this this verdict, which is positively huge in itself. But even more interesting is the why , that nobody seems to have mentioned, which is downright thermonuclear. It becomes clear when you read the detail of the verdict summary, preferably on a good coffee, as I have done:

verdict regarding the Safe Harbor provision, which is a way for US companies to self-report that they meet European requirements for the protection of data concerning individual persons - these data are considered sensitive and worthy of legal protection under European law. However, the verdict name Edward Snowden (!) And what he revealed, saying that companies can promise all they want, but that such promises without effect, that the US authorities (see National Security Agency) and can not replace those promises constantly. The court said that U.S. companies lacking body to contractually guarantee the privacy, US essentially equivalent to minors taken in this regard, unable to contract.

Citing the summary of the verdict, the Court wrote that all on confidentiality contract, promise, or policy "... is only applicable to US companies acceding States and public US authorities are not themselves subject. further requirements, national security, public interest and application of US law outweigh the shelter system, so that US companies are related to a contempt, not limited protection rules under this scheme where they conflict with such requirements. the shelter system for US and allows interference by the US government, with the fundamental rights of individuals ... "

you see how this will greatly beyond the question at the center of Facebook and Google data need to store their data, which was the result reported so far this verdict ?

the full verdict, which was not mentioned at all in the media reports, goes further on this point. It is detailed in paragraphs 73 and forward, with some quotations:

86. Thus, [it is established in Decision 00/520] that "national security, public interest, or the application of laws requirements "take precedence over the principles of the safe harbor, the rule under which the US independent certified organizations receiving personal data from the European Union are required to ignore these principles, but are limit, as they are contrary to these requirements and thus be incompatible with them.
87. In light of the general nature of the derogation provided for in the fourth paragraph of Annex I to Decision 00/520, that decision thus enables interference, based on the requirements of national security and of public interest or the US national legislation with the fundamental rights of persons whose personal data is or could be transferred from the EU to the United States. To establish the existence of an interference with the fundamental right to respect for private life, it does not matter whether the information in question relating to privacy is sensitive or if the persons concerned have suffered consequences negative because of this interference.
94. In particular, the legislation allowing public authorities to have access on a widespread basis the content of electronic communications should be considered as an attack on the essence of the fundamental right to respect for private life guaranteed section 7 of the Charter [of Fundamental Human Rights].

Now, this does not mean that the court has invalidated all contracts between a European entity and a US entity when the US entity has a commitment to privacy. But because the courts do not work like that - they can decide on the very specific matter before them, or they would have legislative power. In that verdict, they declared the Safe Harbor agreement null and void. But the court made clear his reasoning, reasoning that will be applicable to other cases brought before it.

So in practice, the European Court stated that US entities have any agency to promise anything with respect to the confidentiality safeguards, as long as the NSA is operating. Essentially, it held that U.S. companies do not enjoy freedom of contract in the area of ​​privacy, insofar European jurisdiction is concerned.

And it is enormously further than just the Safe Harbor being declared null and void.

Privacy remains in effect your own responsibility. This demonstrates why "trust" should not be a factor of privacy in the first place -. The preferred way is the case with companies that do not need all your data, so no confidence necessary

The Snowden legacy is subtle, but in Full Swing: Encryption Anywhere -

the biggest fear of Edward Snowden was that his sacrifice has no effect. Rather, it is having an effect throughout the whole of the Internet - and the biggest effect in the parts of the net, you will never notice or

They say a good sysadmin is like a window :. If they function optimally, you do not notice them. Encryption is very similar, in that you barely notice it's there, yet it needs to protect your privacy. There is the conflict between the ubiquitous convenience and safety - and until a few years ago, the convenience (and cost) had won this battle. . No more

You can see in the launch of new messaging services: pre-Snowden, nobody asked for encryption. Now it is a point of sale everywhere. In addition, large data centers did not use to encrypt their internal links between data centers - until it was discovered that the unprotected internal links were a primary source of wiretapping to NSA. Now it seems that everyone encrypt internal datalink same.

The important thing here is the place where your loyalty, if you are a data provider or service provider. Do your customers, who pay your salary, or is that your government, which ... well, doing something else? More and more IT companies choose to side with their clients, and necessity, which also ranking against their government. Governments are obviously not happy with this.

Before 2010, you would rarely if ever see the government of a service provider in the threat model against a service. Today, you are not a serious matter if you do not take into account the adversarial governments as part of your threat model against your clients.

This is the context where governments beginning to require any backdoors in encryption. They essentially a confused occasional capacity for the whole world for a wiretap right in this capacity , which are two entirely different things, and they are the least threat to get strength Their path. David Cameron of the United Kingdom may be the most important example, which is arguing that there should be no encryption that the government can not break, and that - as a result - is completely incompetent brand for all technical issues with a unified computing industry. It is good to see that more and more IT companies are taking this kind of position for their clients - for us. Ultimately, it is a struggle governments can not win: if the central departments are obliged to comply, encryption will simply move to the edges, users

Encryption has grown from a niche curiosity to be taken seriously. in just a few years. And everyone is building more.

Privacy remains your own responsibility.

2016 Outlook: Policymaking and technology will continue to diverge -

technology of Policymaking and diverged over the past 30 years, in the battle between a "right to tenure" imaginary one hand and disruptive makers on the other. Eventually, one side has to give.

optimism technology is everywhere. If it is a sentiment that is widespread among diehardest technology entrepreneurs, it is somehow the laws do not apply to their particular masterpiece, as they do all the obnoxious laws useless anyway with the code they write. this goes Uber (which succeeded in several places with that attitude) with Aereo (which did not). what is interesting here is not that some succeed and others not, but the general attitude that the world is changing so fast that the laws are left anyway and is a matter of construction or being outbuilt and you can outbuild the legal framework as anything else.

Meanwhile, many historical industries woke up to the Internet as a threat to their business model, or at least control their aspirations. The industries with ties to the development of policies of innovation diverted resources to write their tenure in the law, which is never a particularly successful business model in the long term. Most of these industries - the copyright industry in particular - seems to truly believe that they are only allowed to punish the future a little longer, a magical unicorn appear that will save this industry for the transition time. This was not a very effective strategy for industries in the past, or to all holders of power. However, they cause little damage to the environment of their development capacity in their attempts to compete using batons and courtrooms instead of trying to compete with the best products and services.

In other words, we can observe that the existence of the popular Internet, people policy attempted to tame technology using the law and politics, and technology managers have tried to tame the policy by using technology. Neither succeeded particularly well, and as they say :. If you keep doing what you did before, he will probably go as he went before

As far back as the mid 1980s, when politicians have tried to legislate the owner's responsibility to operations Bulletin Board Systems (think pre-Internet discussion forums), making sysadmins responsible speech communicated among others on the board (yes, really, and yes, the law is still in force), people understand the technology were nodding in total cluelessness in policymaking. That has not changed in 30 years, with candidates today for the position of president of technology industries asking the US to invent cryptography that can not be circumvented by some people. As someone who is able to tie their shoelaces without supervision can attest, mathematics is inherently unable to work for one person and not to work for someone else. Yet these are the people who run for access to the largest collection in the world with nuclear warheads. It is an understatement to say that frustration in the technology camp was set up for a long time

The same frustration exists in policy development and the camp is the responsibility of the industry. "Why do not obey the laws of technology we do?". But as Jan Carlzon, then-CEO of Scandinavian Airlines observed in the 1980s while turning the company around, "policy beats the market, but the technology beats policy." This illustrates the problem rock -paper-scissors powerplay: if you are a technology company, you are a market player (which can be beaten by policy) or a technology player (beating policy) L? impression remains that most startups themselves as purely technological players before the commercial side of operations hits like a ton of bricks.

This is also the reason why free software and open -source movements really do not care about the laws; they do not have this side of the business that is beatable by politics is why you will always have strong encryption, not just as the open VPN technology today '. hui implementable by anyone with a passion for privacy, but also as Tor and Signal.

there

Ten years, it was observed that the first movement successfully cross the bridge between technology and policy and work successfully from two camps would win the world. The copyright industry has tried to establish a legal bridge to the camp of technology with the DMCA, EUCD and similar laws, which made it illegal to use the way of the technology industry's right to author has not approved - a law which the world almost all ignored since its establishment until now, so you can not really tell the copyright industry worked their way outside the laughingstock status. No such dual player camp emerged as dominant, not for now.

The outlook for 2016 would be as tensions continue to rise and will continue to do so until we have policymakers who understand the Internet. Judging by the age of political candidates, it's not likely to happen by itself until the people who are born with the Internet are of the age of the candidates, which puts us well in the years 2050. the field is ripe for those who want to disrupt this observation.

Linux networking stack from the ground, part 4 -

Part 1 | Part 2 | Part 3 | Part 4 | Part 5

Overview

This post will pick up where the left part 3 begins by describing Receive Packet Steering (RPS), what it is and how to set up, followed by a review the network stack describing how packets are processed according to RPS parameters, packet backlog queue, the beginning of the IP layer, and netfilter.

Receive Packet Steering

We have seen that the device drivers register NAPI polling instances. Each instance NAPI poller runs in the context of a kernel thread called softirq which there is a per CPU. The core CPU thread for the hardware interrupt handler runs on is awake / designed for use in the hardware interrupt handler.

Thus, one CPU processes the interrupt hardware and polls the network layer to process the incoming data.

Some NPI supports multiple queues at the hardware level. This means that incoming packets can be DMA'd to separate receive rings, each ring having received its own hardware interrupt is delivered to indicate the data is available. Each of these hardware interrupts would plan NAPI polling instances to run on each of the associated processors.

This allows multiple processors to handle hardware interrupts and poll the network layer.

Receive Packet Steering (RPS) is a software implementation of hardware enable NPI multi-queue. It allows multiple processors to handle incoming packets, even if the network adapter supports a single queue to get into the hardware.

RPS works by generating a hash for incoming data to determine which CPU must process the data. The data is then queued to the per-CPU receiving network backlog to deal with. An inter-processor interrupt is delivered to the CPU owns the backlog. This will restart the processing of the backlog by the remote CPU if it is not currently processing packets.

netif_receive_skb will either continue to send data over the network to the network stack, or rely on RPS for treatment on another CPU.

RPS set

RPS to work it must be enabled in the kernel configuration (it's on Ubuntu 3.13.0 for Linux kernel), and a bit mask describing the processors must treat packets to an interface and rx given queue.

The bit masks to modify are in / sys / class / net / DEVICE_NAME / files / tail / rps_cpus .

So, for eth0, and receive queue 0, you must change: / sys / class / net / eth0 / files / rx-0 / rps_cpus with a hexadecimal number indicating which processor should process packets eth0 receive queue 0.

Back to netif_receive_skb .

netif_receive_skb

to recall netif_receive_skb function is called from napi_skb_finish in the context of softirq the poller NAPI recorded by the device driver.

netif_receive_skb will either attempt to use RPS (as described above) or keep sending data to the network stack

Let consider to first the second path: .. sending data until the battery if RPS is off

netif_receive_skb without RPS

netif_receive_skb calls __ netif_receive_skb who did some accounting before calling __ netif_receive_skb_core to move the data along to the network stack to the protocol levels.

__ netif_receive_skb_core

This function changes the skb to the protocol layer in this piece of code (net / core / dev.c: 3628):

 skb- type => Protocol; list_for_each_entry_rcu (ptype, & ptype_base [ntohs(type) & PTYPE_HASH_MASK], list) {if (ptype-> type == type && (ptype-> dev == || null_or_dev ptype-> dev skb- ==> || dev ptype-> = dev orig_dev =)) {if (pt_prev) ret = deliver_skb (skb, pt_prev, orig_dev); pt_prev = ptype; }} 

We'll look at exactly how this code provides data to the protocol layer below, but first, let's see what happens when RPS is on.

netif_receive_skb RPS

RPS If enabled, netif_receive_skb calculate that the CPU's backlog, he must queue data. It does this by using get_rps_cpu (defined in net / core / dev.c: 2980)

 int cpu = get_rps_cpu (skb-> dev, skb, & rflow); if (cpu> = 0) {ret = enqueue_to_backlog (skb, cpu, & rflow-> last_qtail); rcu_read_unlock (); return ret; } 

enqueue_to_backlog

This function first get a pointer to softnet_data structure of the remote CPU that contains a pointer to a poller NAPI.

Then, the length of the queue of waiting input_pkt_queue to the remote CPU is satisfied:

 = qlen skb_queue_len (& SD-> input_pkt_queue); if (qlen <= netdev_max_backlog && !skb_flow_limit(skb, qlen)) { if (skb_queue_len(&sd-> input_pkt_queue)) {

There is first with respect to the netdev_max_backlog . If the length of the queue is larger than the order backlog, the data is deleted and the fall is counted against the remote CPU

You can prevent falls by increasing netdev_max_backlog :.

 sysctl -w net.core.netdev_max_backlog = 3000 

If the length of the queue is not too large, the next code checks if the flow limit is reached . By default, the speed limits are disabled. To enable rate limits, you must specify a bitmap (similar to RPS bitmap) in / proc / sys / net / core / flow_limit_cpu_bitmap .

Once you have activated the CPU speed limits, you can also adjust the flow limit of the hash table size by modifying the sysctl net.core.flow_limit_table_len .

you can learn more about the flow limits in Documentation / networking / scaling. txt file.

assuming that the flow limit has not been met, enqueue_to_backlog then checks if the backlog queue has data queued in already .

If so, the data is queued:

 if (skb_queue_len (& SD-> input_pkt_queue)) {enqueue: __skb_queue_tail (& SD-> input_pkt_queue, skb); input_queue_tail_incr_save (sd, qtail); rps_unlock (nd); local_irq_restore (flags); NET_RX_SUCCESS return; } 

If the queue is empty, the first NAPI poller for the backlog queue is launched:

 / * NAPI schedule for the delay device * We may use non-atomic operation because we have the lock of the queue * / if (__ test_and_set_bit (NAPI_STATE_SCHED, & SD-> backlog.state!)) {if ____ napi_schedule (sd, & SD-> backlog) (rps_ipi_queued (nd)!); } Goto enqueue ;. 

The goto at the bottom brings execution was above the code block, the queue data to the backlog

backlog queue, NAPI waiting poller

queue backlog by CPU plugs into NAPI in the same way a device driver is. A sampling function is provided that is used to process packets from softirq context.

This struct NAPI is provided during initialization of the networking system. Of net_dev_init in net / core / dev.c: 6952:

 SD-> = backlog.poll process_backlog; SD-> = backlog.weight weight_p; SD-> backlog.gro_list = NULL; SD-> backlog.gro_count = 0; 

The structure of the NAPI backlog NAPI structure differs from the device driver in the weight parameter is adjustable. Pilots hardcode values ​​(more hardcode to 64, as seen in e1000e).

Set NAPI weight poller backlog, change /proc/sys/net/core/dev_weight.

The poll function for the backlog is called process_backlog , and, similar to the function of e1000e e1000e_poll , is called from the context of softirq.

process_backlog

The process_backlog (net / core / dev.c: 4097) is a loop that runs until its weight (specified in `/ proc / sys / net / core / dev_weight`) was consumed or no more data remains on the order book.

Each piece of data waiting in the backlog queue is removed from the queue backlog and forwarded to __ netif_receive_skb . As explained above in any case of RPS, the data passed to this function finally reaches protocol layers after some accounting.

Similarly NAPI device driver implementations, the process_backlog code disables the poller if the total weight will not be used. The voter is restarted with the call to ____ napi_schedule of enqueue_to_backlog as described above.

The function returns the amount of work done, which net_rx_action (described above) will subtract from the budget (which is adjusted with the net.core.netdev_budget as described above).

__ netif_receive_skb_core provides data on protocol layers

The __ netif_receive_skb_core provides data on the protocol layers. It does this by getting the protocol field of skb and iterate over a list of provide functions registered for this type of protocol.

What is happening in this piece of code (as above shown):

 skb- type => Protocol; list_for_each_entry_rcu (ptype, & ptype_base [ntohs(type) & PTYPE_HASH_MASK], list) {if (ptype-> type == type && (ptype-> dev == || null_or_dev ptype-> dev skb- ==> || dev ptype-> = dev orig_dev =)) {if (pt_prev) ret = deliver_skb (skb, pt_prev, orig_dev); pt_prev = ptype; }} 

The ptype_base ID is defined in level net / core / dev.c: 146as a hash of lists:

 struct list_head ptype_base [PTYPE_HASH_SIZE] __read_mostly ; 

Each protocol layer adds struct packet_type to a list in a specific location in the hash table.

slot in the hash table is calculated by ptype_head

 static inline struct list_head ptype_head * (const struct packet_type * pt) {if (PT> Type == htons (ETH_P_ALL)) return & ptype_all; Another back and ptype_base [ntohs(pt->type) & PTYPE_HASH_MASK] } 

The protocol layers call dev_add_pack to add to the list.

layer IP

The IP protocol layer attaches to the ptype_base hash table so that data will be delivered to the lower layers it

This occurs in the inet_init net / ipv4 / af_inet.c :. 1815

 dev_add_pack (& ​​ip_packet_type); 

This stores the IP packet type structure defined as follows:

 static struct packet_type ip_packet_type __read_mostly = {.type = cpu_to_be16 (ETH_P_IP) .func = ip_rcv,}; 

__ netif_receive_skb_core calls deliver_skb (as seen in the above section). This function (net / core / dev.c: 1712)

 static inline int deliver_skb (struct sk_buff * skb, struct packet_type pt_prev *, struct * net_device orig_dev) {if (unlikely (skb_orphan_frags (skb, GFP_ATOMIC))) return -ENOMEM; atomic_inc (& skb-> users); return pt_prev-> func (skb, skb-> dev, pt_prev, orig_dev); } 

In the case of IP, the ip_rcv function is called.

ip_rcv

The ip_rcv function is pretty simple at a high level. There are several integrity checks to ensure data validity. Statistics counters that are superseded and

. Ip_rcv ends by passing the package ip_rcv_finish through netfilter. This is done so that all the iptables rules that should be matched to the IP protocol layer can have a look at the package before it continues (net / ipv4 / ip_input.c: 453):

 back NF_HOOK (NFPROTO_IPV4, NF_INET_PRE_ROUTING, skb, dev, null, ip_rcv_finish); 

netfilter

The NF_HOOK_THRESH is pretty simple. It calls to nf_hook_thresh and the success, called okfn which in our case is ip_rcv_finish (include / linux / netfilter.h: 175):

 static inline int nF_HOOK_THRESH (uint8_t pf, unsigned int hook struct sk_buff * skb, struct net_device * in, struct net_device * out, int (* okfn) (struct sk_buff *), int thresh) {int ret = nf_hook_thresh ( pf, hook, skb, inside, outside, okfn beat); if (ret == 1) ret = okfn (SKB) return ret; } 

The nf_hook_thresh function

continues down iptables approach. It begins by determining if the netfilter hooks to the chain of netfilter protocol family and netfilter transmitted.

In our example above, the protocol family is NFPROTO_IPV4 and the type of chain is NF_INET_PRE_ROUTING

 / ** * nf_hook_thresh - call a netfilter hook * * Returns 1 if the hook has the package to pass. * Okfn the function must be invoked by the appellant in this case. Any other return value * indicates that the packet was consumed by the hook. * / Static inline int nf_hook_thresh (u_int8_t pf, unsigned int hook struct sk_buff * skb, struct net_device * Indev, struct net_device * outdev, int (* okfn) (struct sk_buff *), int thresh) {if (nf_hooks_active (pf, hook)) return nf_hook_slow (pf, crochet, skb, Indev, outdev, okfn, thresh); return 1; } 

This function calls the function nf_hooks_active which examines a table called nf_hooks_needed (include / linux / netfilter.h: 114):

 static inline bool nf_hooks_active (u_int8_t pf, unsigned int hook) {return list_empty (& nf_hooks [pf] [hook]) !; } 

And if this one hook, nf_hook_slow is called to go further in iptables.

nf_hook_slow

nf_hook_slow through the list of hooks in nf_hooks table for the protocol type and the type of chain by calling nf_iterate for each entry in the list of hook.

nf_iterate in turn calls the hook function associated with an entry on the hook list and returns a "verdict" on the package.

iptables ... Tables

iptables saves the hook functions for each of the corresponding tables packages :. filter, nat, mangle, raw, and security

in our example, we are interested in NF_INET_PRE_ROUTING chains that are in the nat Table [

Indeed, the struct pointer with the hook function that is registered with netfilter is in net / ipv4 / netfilter / iptable_nat.c: 251

 static struct nf_hook_ops nf_nat_ipv4_ops [] = {__read_mostly / * Before packet filtering, change of destination * / = {.hook nf_nat_ipv4_in, .owner = THIS_MODULE, .pf = NFPROTO_IPV4, .hooknum = NF_INET_PRE_ROUTING, .Priority = NF_IP_PRI_NAT_DST,}, 

, which is part of iptable_nat_init (net / ipv4 / netfilter / iptable_nat.c: 316):

 err = nf_register_hooks (nf_nat_ipv4_ops, ARRAY_SIZE (nf_nat_ipv4_ops)); if (err <0) goto err2; 

In our example above the IP layer, the packets will be forwarded to nf_nat_ipv4_in down further in iptables via nf_hook_slow function described in the previous section.

nf_nat_ipv4_in

nf_nat_ipv4_in passes the package on nf_nat_ipv4_fn which begins by obtaining information conntrack for the package:

 struct nf_conn * ct; enum ip_conntrack_info ctinfo; / * Slightly abridged sample code * / ct = nf_ct_get (skb, & ctinfo); 

If the package under discussion is a package for a new connection, the nf_nat_rule_find is called (net / ipv4 / netfilter / iptable_nat.c: 117):

 IP_CT_NEW case: / * seen before? This can happen for resupply, retrans, * or local packages. * / If (nf_nat_initialized (ct, maniptype!)) {Unsigned int ret; ret = nf_nat_rule_find (skb, PAHO, and finalize> hooknum, in, out, ct); if return ret (ret = NF_ACCEPT!); 

And, finally, nf_nat_rule_find calls ipt_do_table entering the iptables subsystem. This is as far as we'll get into the Netfilter and iptables systems because they are complex enough to warrant their own multi-page documents.

The return value of the function ipt_do_table will be

• not NF_ACCEPT , in which case it is returned immediately, oR
• will be NF_ACCEPT causing nf_nat_ipv4_fn to call nf_nat_packet for handling packets and return either NF_ACCEPT or NF_DROP .

unwinding the return value

In both cases the return value for ipt_do_table , the final value of nf_nat_ipv4_fn is returned back through all the functions described above until NF_HOOK_THRESH

1. nf_nat_ipv4_fn the return value is returned nf_nat_ipv4_in
2. that returns to nf_iterate
3. that returns to nf_hook_slow
4. that returns to nf_hook_thresh
5. that returns to nF_HOOK_THRESH

nF_HOOK_THRESH checks the return value and if it is NF_ACCEPT (1), it calls the function pointed by okfn .

In our example, okfn is ip_rcv_finish that will be part of the treatment and forwards the packet to the next protocol layer.