I've not been able to connect to newsgroups all day. The ES web site is showing a fraction of the usual number of sessions. This happened a while back and word was that they were attacked. I guess someone is back at it again?
Rick
Not for me, but other services note higher than normal customer enquires needing attention.
Maybe it is simply the time of year?
Given that, I frequently have troubles connecting to ES, but it comes back within a few hours.
Very goods for a free service ;o)
Grant.
Maybe they are based in North Korea...
Normally, that would be considered a good thing. Perhaps reading a book or doing something useful would be a suitable alternative? I stayed home today to catch up on my billing and taxes. Instead, I spent the day oversleeping and watching a video. It's a tough life, but I can survive without my daily does of Usenet.
Yep. There seems to be a problem that started last night. There are usually about 1500 sessions. Now, there are only 200. Normal seems to be about 600 sessions. Also, the news portal seems to be offline.
Traceroute to news.eternal-september.org jumps from inside Cogent from
90 to 160 msec, which is probably where things are slowing down. However, 160 msec from the US left coast to Germany is high, so I don't think it's a DOS attack. Might be because there's currently a problem between Nova Scotia and UK on the way to Germany.Looking at the bottom MRTG graph at: It looks like someone is generating an ever increasing number of bogus connections. Note how the number of connections increases over the month until it hits a peak, where they are cleared probably by the admins, and the number again begins to ramp up. That's NOT a normal usage pattern, which suggests either an attack or an NNTPD process that is not closing connections properly. I can't tell from here but that doesn't look like an attack, which would show an abrupt increase, not a slow ramp up.
The first step to solving a problem is to blame someone. Now that the preliminary blame has been established, we can move on to the punishment of the innocent and the promotion of the guilty.
-- Jeff Liebermann jeffl@cruzio.com 150 Felker St #D http://www.LearnByDestroying.com Santa Cruz CA 95060 http://802.11junk.com Skype: JeffLiebermann AE6KS 831-336-2558
Until now... obviously you couldn't hold out the full day. The first step is admitting you have a problem.
I don't follow that at all. These sorts of attacks are often done by spreading a virus which then attacks the target. I can see where they might not all go off at the same time either by design or by accident.
Rick
showing a fraction of the usual number of sessions. This happened a while back and word was that they were attacked. I guess someone is back at it a gain?
Anyone know why Thunderbird is so spastic when it can't contact the server? I try to look at an old message and it zooms off to another group. I try to look at a message in that group and it zooms off to another group again . I have to switch it over to offline use to view any old messages and eve n then most of them aren't there in spite of the fact that I've told T-bird to not delete any messages.
Rick
Yeah, I know. I've been on Usenet for far longer than I care to admit. Like eating, drinking and breathing, it's an addiction that's not easily eliminated. I did try to survive a weekend without the internet, but failed, and never tried again.
That's possible, but not the way it's usually done. More commonly, the controller fires off all the bots under their control at once. That's because if the internet becomes congested, they may have problems regaining control, and that some users are bound to notice causing a decrease in DDOS traffic. The usual traffic pattern is an abrupt increase in traffic, followed by a slow decline as ISP's block traffic from the higher traffic sites. Eventually, the traffic drops to normal.
Various sites monitor DDoS related traffic. For example: (give it some time to load).
In this case, the Sony hack was discovered on Nov 24. Any retaliatory traffic prior to that date is unlikely to have been involved with the Sony hack. On the news.eternal-september.org connection graph at: Notice that the "ramp" like waveforms started in early October, which suggests that the increasing number of connections are probably not related to the Sony attack 6 weeks later.
However, it is interesting that the current drop in the number of connections coincides with the beginning of the North Korean DDoS attack at 0300 UTC on Monday. There just might be a connection. Also note that the news server load average is very low, meaning that it's not getting much work to do from user connections. 'Eternal September News Server' had been up for 03:05:02 up 289 days, 11:15, 0 users, load average: 0.00, 0.03, 0.05.
-- Jeff Liebermann jeffl@cruzio.com 150 Felker St #D http://www.LearnByDestroying.com Santa Cruz CA 95060 http://802.11junk.com Skype: JeffLiebermann AE6KS 831-336-2558
Been in NK lately?
Jamie
Been opening buffer overrun Jpegs lately or even hovering over a couple in OutLook Depress ?
Jamie
Update. The shutdown of connectivity to North Korea had nothing to do with the loss of connectivity to Eternal-September. Looks Eternal-September was having some kind of problem. Note the monthly graph for week 51: The number of NNTP sessions just kept increasing and increasing until it hit 1500 sessions, where it flat topped. I think this is the point where new connections were failing, where the server ran out of sockets, inodes, streams, buffers, CPU cycles, patience, or whatever. Looks like they cleaned up the mess on Monday morning, which by coincidence was when North Korea went off line. They then seem to have been floundering around until mid Tuesday, where things returned to normal with about 650 to 750 sessions. It's too soon to tell if the number of NNTP sessions will start to climb again, but it looks good so far for the last 4 days.
-- Jeff Liebermann jeffl@cruzio.com 150 Felker St #D http://www.LearnByDestroying.com Santa Cruz CA 95060 http://802.11junk.com Skype: JeffLiebermann AE6KS 831-336-2558
Thank you for posting this, I have problems with my ISP downgrading nntp traffic to low priority, so I'm used to fairly poor performance of their site.
Grant.
FWIW, my newsreader always asks for news.eternal-september.org . When things are working normally, I see that same hostname as a banner when my newsreader starts up. During the recent difficulties, I saw mx02.eternal-september.org in the banner instead.
At first, I could always connect, but things were slow. Then, my initial connection attempt would be refused, and I would need to retry several times (like, 5 to 15) to get a connection. Once I got a connection, things proceeded at more or less the normal speed. This retry-until-it-works was when I saw mx02 answering the phone.
My guess is that something went wrong with the "main" server(s), so operations were shifted to backup server(s), with a lower administrative limit on the number of connections.
As I write this, things seem to be working OK.
Matt Roberds
I'd be willing to bet they were under attack again, but it's hard to say unless they provide some insight. I'm connected to ES all the time but only post or read messages a few times a day for a few minutes each time. I might not notice if the connection were a bit slow. The first I realized I couldn't get a connection at all. Thunderbird gets very weird with the focus jumping randomly between groups when that happens, so it is very obvious. That was the first sign I noticed. Once it was down, it was down solid for me for better part of the day. By that time the connection rate had peaked and slammed down to low values.
-- Rick
...
That pretty sums it up for my way of working, kick Agent into reading selected newsgroups, minimise that window and go on with other stuff, I might check some hours later, this IS a low priority information stream (as far as timing goes), after all.
It's them people downloading nntp movie binaries choking the 'net these days? Or the new generation who know not this old protocol?
Grant.
mx02.eternal-september.org seems to be the backend server for their news feed. If you look at the path line in the NNTP header, you'll see something like: and: So, you post to mx02 and read from what I would guess is a load balancing router feeding multiple servers under eternal-september.org. Sorry, but I don't have any topology info for them.
Looking at the NNTP connections graph from last Monday and Tuesday, it looks like they were juggling reader servers for a day, until they fixed whatever was causing the number of NNTP connections to constantly increase.
Note the flat tops for Week 51 in: That means the no new connections over 1500 were being allowed. The servers weren't particularly busy, so it wasn't like they were being attacked or overloaded. They're using INN, so my guess(tm) is they just ran out of
I don't think so. Again, looking at the continued operation through Monday, I believe that they took down each reader server, one at a time, to do some kind of software change, and then put it back online. The graph seems to indicate only two servers: mail.eternal-september.org news.eternal-september.org but that's a guess, because the load balancer can hide any number of servers behind a single FQDN or IP address. However, judging by the two "lurches" in the graph on Monday, I would guess(tm) that there are only two reader servers involved. Incidentally, note that the uptime is still 294 days, which means they did the changes without rebooting or dropping connected users or losing traffic:
Don't hold your breath. Look at the Monthly graph for Week 52 (this week) and notice that the graph trend is starting to repeat its endless rise again. Oh-oh. I don't think it's fixed.
-- Jeff Liebermann jeffl@cruzio.com 150 Felker St #D http://www.LearnByDestroying.com Santa Cruz CA 95060 http://802.11junk.com Skype: JeffLiebermann AE6KS 831-336-2558
I would really like to take you up on that bet. However, my family has a long history of producing compulsive gamblers. I apparently have the family curse, so I avoid all forms of gambling, including sure things like this bet. Sorry.
See the graph for Week 51 (last week) on the Monthly graph at: and note the "flat tops" on the number of NNTP connections. That means that the server has hit its limit of 1500 connections and is refusing to accept any more. That's why you couldn't get a connection. While taking down North Korean connectivity for a few hours probably involved a DDoS (distributed denial of service) attack, I doubt that it would also be directed against port 119 (NNTP) on a news server located in Germany. The only connection was the coincidence that both happened at the same time.
The bad news is that if you look at Week 52, you'll see that the rising trend is back, and that the problem (whatever it may be) has not been fixed. Looking at the history, it takes 2 to 3 weeks to hit
1500 NNTP connections. Stay tuned.Incidentally, taking down internet access to North Korea isn't going to do anything useful. If we really want to fix the North Korea "problem", we should be air dropping satellite terminals with free internet access so that the people can see what the rest of the world is like.
-- Jeff Liebermann jeffl@cruzio.com 150 Felker St #D http://www.LearnByDestroying.com Santa Cruz CA 95060 http://802.11junk.com Skype: JeffLiebermann AE6KS 831-336-2558
I know what I saw at the time of the problem. The number of connections had risen to nearly 1600 over some period with no problem. When I had a problem getting access was *after* the number dropped well off again and the lack of connection persisted for about half a day after the number had dropped into the low hundreds range. Rather than being a smooth daily wave as you see now, it was a chaotic up and down sometimes very sharp. My outage coincided with the minimum connection span late Monday going into Tuesday on the current weekly graph.
-- Rick
I knew I had seen that somewhere before.
But I *think* that when I connected before, the server would announce itself as news.eternal-september.org. I tried it again just now and it's back to mx02:
$ telnet news.eternal-september.org 119 Trying 213.239.209.88... Connected to news.eternal-september.org. Escape character is '^]'.
200 mx02.eternal-september.org InterNetNews NNRP server INN 2.6.0 (20141222 snapshot) ready (posting ok) QUIT 205 Bye! Connection closed by foreign host.That's a relatively recent version of INN.
...sockets? disk space? inodes? anodes? triodes? NoDoz?
I wonder if some new version of an alleged newsreader (like Outhouse) has been released lately, and it's hanging on to old connections rather than closing them. Or perhaps there is a new version of some popular firewall/routing software that forgot how to NNRP.
Matt Roberds
Sorry. I wasn't sure, did some Googling, didn't find an instant answer, was interrupted by something, and forgot to fix my omission. My guess(tm) is that INN didn't run out of anything. The inn.conf file parameter "maxconnections" is set to 1500 NNTP connections. Once this limit was hit, INN would not allow any additional connections. At some point, the admins probably killed off most of the sessions, thus allowing additional connections.
That's possible but unlikely. Again, take a look at: The Yearly graph shows that the problem started in early October and persisted continuously since then. Unless a huge percentage of the users all switched to a different and buggy newsreader at the same time, I don't see how a buggy newsreader might cause the almost linear increase in NNTP sessions.
Nope. The firewall doesn't keep NNTP sessions open. Only the INN server does that.
I think the best question to ask the Eternal-September admins is "what changed on Oct 1"? I think you may have hit it when you noticed the INN version number. The current stable version is 2.5.4 but Eternal September is running the development 2.6.0 version. Kinda smells like a beta version possibly with bugs:
-- Jeff Liebermann jeffl@cruzio.com 150 Felker St #D http://www.LearnByDestroying.com Santa Cruz CA 95060 http://802.11junk.com Skype: JeffLiebermann AE6KS 831-336-2558
ElectronDepot website is not affiliated with any of the manufacturers or service providers discussed here. All logos and trade names are the property of their respective owners.