Does the extreme violence and sex in Game of Thrones affect its ratings?

Let's look at the ratings! All the numbers in my post are based on the zap2it reports of Nielsen ratings. First, a pretty graph to give you the jist of it (click to enlarge):

As you can see, generally the ratings are rising, however there are several dips:

1. You Win or You Die, S01E07, May 29th 2011, -0.04 (40,000) viewers
2. Blackwater, S02E09, May 27th, 2012, -0.48 (480,000) viewers

These two are important as they're the start of a trend, these are both the Sunday before the Last Monday of May, which is Memorial Day in the USA. Because of these two drops, there was no episode on Memorial Day weekend in 2013 or 2014, so we'll discount them for now. The second set we'll be discounting are the second episodes after a season premiere, as pilots and premiers always draw in new viewers due to the increased advertising they receive and more people trying the show to see if they like it.

1. The Kingsroad, S01E02, -0.02 (20,000) viewers
2. The Night Lands, S02E02, -0.1 (100,000) viewers
   • There were several deaths in the previous episode: An unnamed knight in King's Landing, Rakharo was murdered off-screen, plenty of unnamed Septa were burned to death at Dragonstone, Maester Cressen committed suicide attempting to murder Melisandre and most of Robert Baratheons bastards were murdered, including a baby off-screen. However as mentioned, I don't believe these were the reasons for it, as seasons 2 and 3 tied for lowest IMDb rated pilots (8.8 compared to 8.9 for season 1 and 9.1 for season 4)
3. Dark Wings, Dark Words, S03E02, -0.1 (100,000) viewers
   • Again, this is after the tied-for-lowest premiere, and as detailed with pilots, it may just be most of the +0.17 (170,000) viewers gained compared to the season 2 finale elected not to continue. There were no deaths or sex scenes in the previous episode, although the Warlocks of Qarth attempted to murder Daenerys with a scorpion.
4. Two Swords, S04E02, -0.33 (333,000 viewers)
   • This is the biggest drop for a season premiere, however season 4 also had the largest jump up beforehand of +1.25 (1,250,000) viewers, so it's still on scale, even accounting for the fact this was the highest rated premiere.

Season 1 drops:

• Baelor, S01E08, -0.06 (60,000) viewers
  • There were several notable deaths in the previous episode; Syrio Forel, the entire Stark household, Drogo's Bloodrider Mago, as well as the rape and slaughter of the Lamb-men village.

However, after the following episode in which Eddard Stark is killed, there was no subsequent drop in viewers, and he was certainly (one of) the most liked characters.

Before we get to the season 2 drops, I'd like to show you the IMDb user ratings for the show, courtesy of kevinformatics:

As you can see, the early episodes from every season are poorly received compared to later episodes, and this is the best reason I can find to explain the drops early on in season 2:

• Garden of Bones, S02E04, -0.12 (120,000) viewers
  • In What is Dead May Never Die, most of those heading to Castle Black were murdered, including Lommy. There was also a gay sex scene between Renley and Ser Loras. I don't really want to comment on the motives of people leaving at this point, but there's been plenty of heterosexual sex and violence against children at this point so that's nothing new. Certainly a possibility.
• The Old Gods and the New, S02E06, -0.02 (20,000) viewers
  • In The Ghost of Harrenhal, Renley is murdered by magic in possibly the least violent death yet depicted, this combined with the fact he was a fairly minor character in the show leads me to believe it was not about the violence.
• A Man Without Honor, S02E07, -0.19 (190,000) viewers
  • Ser Rodrik Cassel is beheaded by Theon in Winterfell, quite brutally, The Hound disembowels some men attempting to rape Sansa and Irri dies off-screen.

As you can see, this is the biggest group of losses in the show, a lot of major characters are killed and I'd be willing to believe its because of the violence, but I suspect most people will have realised the tone after Eddard's death. It may be that they did, but were hoping it would change. Regardless, season 3 is much more consistent (other than Blackwater):

• The Bear and the Maiden Fair, S03E07, -0.66 (66,000) viewers
  • The previous episode, The Climb, was the 3rd lowest rated episode and featured no deaths or sex scenes, though Theon's little finger was flayed by Ramsay. It's very possible this was due to the violence as it was the most graphic torture scene thus far and was very uncomfortable to watch. However, there were no viewer drops after the next episode which featured Theon being castrated, which was far worse.

Season 4:

• The Laws of Gods and Men, S04E06, -0.76 (760,000) viewers
  • This is the largest viewer drop off so far, and the lowest rated episode of season 4 (8.8).
  • There's no sex scenes in First of His Name, however Karl Tanner prepares to rape Meera before being impaled through the back of the head by Jon Snow, not particularly graphic compared to previous events.
• The Mountain and the Viper, S04E08, -0.02 (20,000) viewers
  • The Hound killed a dying man, The Mountain killed several prisoners, Petry Baelish kills Lysa Arryn, Melisandre has a topless scene. While these are all possible explanations, I think it's far more likely that people didn't know the show was on, as the previous week didn't have an episode.

Really this is all just speculation, but I don't believe there's any real relationship between sex and violence and viewerships, though I'm certain there are definitely thousands of people who have stopped watching because of it, the overall shift hides it too completely. You also shouldn't forget that this is the internet, and the same thing is likely to attract others for the same reasons, internationally and illegally it's doing amazingly well too.

The last episode was very violent compared to most, so if we see a huge dip that may provide an answer, though even after episodes like The Rains of Castamere when the Starks were finally defeated in a very brutal manner, the increase in viewership was substantial.

tl;dr: Nope