1. Euphoria website down, members being lost

The website being down constantly is losing us members.
http://sourceforge.net/tracker/index.php?func=detail&aid=2868567&group_id=182827&atid=902782

It is currently not possible to sign up at all:
http://openeuphoria.org/ticket/82.wc

Anyone else in favor of a new server and some thorough site debugging?

new topic     » topic index » view message » categorize

2. Re: Euphoria website down, members being lost

I guess you don't understand the ramifications of the 46058 views counted?

At 2:07 pm today, that topic had been "viewed" 15281 times, and by 7:33pm today it had been "viewed" 46033 times. Do you think it's the host's problem that in 5.5 hours the one topic had been successfully "viewed" 30752 times, or 93 times per minute? Note it didn't count the unsuccessful "views", which could easily have been 10 times that figure. At 93 hits per second, it's unlikely the "viewer" will complete the page fetch to the point of getting the "view" counter incremented, but the site will still be sustaining (or not) an abnormal load. Caveats implied as needed.

I agree these aren't huge numbers, but we don't have all the numbers, and obviously the site doesn't have effective anti-dos code running. Maybe once the host begins tacking on financial surcharges for the bandwidth?

useless
Forked into: Time for Pagination?

new topic     » goto parent     » topic index » view message » categorize

3. Re: Euphoria website down, members being lost

" OM3GA Sentinel - Anti-DoS & DDoS 1.6 description Once you follow the simple installation procedure, you will be protected. The script will monitor your server load. Once your load hits 20 your site will automatically be disabled from processing php to prevent your server from crashing. At that point it will begin logging IP address that attempt to access the downed page. By using this method, OM3GA SENTINEL will stabalize your server load, regardless of how long you are being attacked. This program also has a very nice admin panel which can be viewed during or after an attack. It will display a neat log of all the IPs that attempted to access the downed page, and the number of times they tried to access it. The attackers IP(s) are listed at the top of the page, so you can easily report them to thier ISP, and/or block thier IPs from accessing your page."

try that if its bad.

new topic     » goto parent     » topic index » view message » categorize

4. Re: Euphoria website down, members being lost

Fendaril said...

" OM3GA Sentinel - Anti-DoS & DDoS 1.6 description ..."

try that if its bad.

Protection is in place however the site shouldn't have had a problem. I stress tested euweb at 200 requests a second with no problem. Our web host, however, doesn't want 200 processes running for just one user, so they limit it. Thus, when 200 requests are going on at the same time, a large percentage of those are getting 500 errors because the web server refuses to start another process.

Jeremy

new topic     » goto parent     » topic index » view message » categorize

5. Re: Euphoria website down, members being lost

jeremy said...

Protection is in place however the site shouldn't have had a problem. I stress tested euweb at 200 requests a second with no problem. Our web host, however, doesn't want 200 processes running for just one user, so they limit it. Thus, when 200 requests are going on at the same time, a large percentage of those are getting 500 errors because the web server refuses to start another process.

Jeremy

Sounds to me like a good reason to move to a new web host. Should we hold a vote (on the forums) for it?

new topic     » goto parent     » topic index » view message » categorize

6. Re: Euphoria website down, members being lost

jimcbrown said...
jeremy said...

Protection is in place however the site shouldn't have had a problem. I stress tested euweb at 200 requests a second with no problem. Our web host, however, doesn't want 200 processes running for just one user, so they limit it. Thus, when 200 requests are going on at the same time, a large percentage of those are getting 500 errors because the web server refuses to start another process.

Jeremy

Sounds to me like a good reason to move to a new web host. Should we hold a vote (on the forums) for it?

Won't do any good. Every host will impose limit or surcharge for bandwidth.

jacques

new topic     » goto parent     » topic index » view message » categorize

7. Re: Euphoria website down, members being lost

jacques_desch said...

Won't do any good. Every host will impose limit or surcharge for bandwidth.

We are no where near a bandwidth limit.

Jeremy

new topic     » goto parent     » topic index » view message » categorize

8. Re: Euphoria website down, members being lost

jacques_desch said...
myself said...

Sounds to me like a good reason to move to a new web host. Should we hold a vote (on the forums) for it?

Won't do any good. Every host will impose limit or surcharge for bandwidth.

jacques

My understanding is that we are on a shared host with rather strict limits (i.e. on the amount of memory per process, number of processes running simultaneously, etc) that would not exist on a more dedicated host, and it is these limitations that are causing us issues with 500 errors and the site being down.

The bandwidth limitation (and other fundamental limitations such as total memory and cpu usage for the entire system, disk space, etc) we do not have an issue with. So if we move hosts, we'd be fine.

new topic     » goto parent     » topic index » view message » categorize

9. Re: Euphoria website down, members being lost

jeremy said...
jacques_desch said...

Won't do any good. Every host will impose limit or surcharge for bandwidth.

We are no where near a bandwidth limit.

Jeremy

But we are reeling from the cpu usage limits of the system. (We aren't causing the cpu issues, but we are reeling from them regardless.)

new topic     » goto parent     » topic index » view message » categorize

10. Re: Euphoria website down, members being lost

Regardless that just means the bot has to run longer Jim.

new topic     » goto parent     » topic index » view message » categorize

11. Re: Euphoria website down, members being lost

Fendaril said...

Regardless that just means the bot has to run longer Jim.

With a better host, we should be able to effect improved countermeasures against these attacks without compromising the service of the forum.

new topic     » goto parent     » topic index » view message » categorize

12. Re: Euphoria website down, members being lost

I notice that its still impossible to create an account, now due to coding errors.

./../euweb/edbi/edbi.e:392 in function last_insert_id()  
type_check failure, seq_name is 5  
... 

Also, valid email addresses are rejected.

Also, its still not possible to stay logged in when using a dynamic IP.

How, exactly, does Eu intend to grow if new members can't sign up?!

new topic     » goto parent     » topic index » view message » categorize

13. Re: Euphoria website down, members being lost

jimcbrown said...
Fendaril said...

Regardless that just means the bot has to run longer Jim.

With a better host, we should be able to effect improved countermeasures against these attacks without compromising the service of the forum.

Then I am in support for a new host. The problem is money and support at this point.

new topic     » goto parent     » topic index » view message » categorize

14. Re: Euphoria website down, members being lost

Fendaril said...
jimcbrown said...
Fendaril said...

Regardless that just means the bot has to run longer Jim.

With a better host, we should be able to effect improved countermeasures against these attacks without compromising the service of the forum.

Then I am in support for a new host. The problem is money and support at this point.

So why not a simple php/euphoria script that disallows more than a few requests a minute per ip, only non-duplicate requests, and only one request at a time? Robsz and Unkmar and i did this in 1999(?) for something we were working on in Euphoria for the internet. We returned a shortened header and 3 words, or less traffic than the simulated attacker would be sending. Or you could just drop the connection.

useless

new topic     » goto parent     » topic index » view message » categorize

15. Re: Euphoria website down, members being lost

useless said...

So why not a simple php/euphoria script that disallows more than a few requests a minute per ip, only non-duplicate requests, and only one request at a time?

useless

We could do this in pure euphoria. That would stop the DOS attacks and is a good idea and should be implemented right away.

But it won't stop the 500 errors, for reasons I've already outlined above. (To repeat them, the host has capped the number of processes we can run, so if 101 users want to look at the forum at the same time, one of them will see a 500. Also, there is a user on the host who runs "exu ./eu.cgi" using 100% of cpu which then causes the host to not allow us to start any processes.) So I feel we still need to move to a new host.

new topic     » goto parent     » topic index » view message » categorize

16. Re: Euphoria website down, members being lost

Fendaril said...

Regardless that just means the bot has to run longer Jim.

No, not really. It just means we have to have a better host that realizes a DOS attack is going on and takes care of it as they should. It's not our fault we have a DOS attack going on and it's eatting their bandwidth affecting hundreds of their users.

Simply put, they care less. They are probably a low budget host who puts a computer on the net, collects money and forgets about it.

Jeremy.

new topic     » goto parent     » topic index » view message » categorize

17. Re: Euphoria website down, members being lost

useless said...

So why not a simple php/euphoria script that disallows more than a few requests a minute per ip, only non-duplicate requests, and only one request at a time?

This was done a few days ago. It's just a matter of not developing things you don't need. We never needed it, so it was never developed. Now that it is needed, it was developed and deployed.

Jeremy.

new topic     » goto parent     » topic index » view message » categorize

18. Re: Euphoria website down, members being lost

jeremy said...
Fendaril said...

Regardless that just means the bot has to run longer Jim.

No, not really. It just means we have to have a better host that realizes a DOS attack is going on and takes care of it as they should. It's not our fault we have a DOS attack going on and it's eatting their bandwidth affecting hundreds of their users.

Simply put, they care less. They are probably a low budget host who puts a computer on the net, collects money and forgets about it.

Jeremy.

Would getting either a dedicated server, or colocation hosting be an option?

new topic     » goto parent     » topic index » view message » categorize

19. Re: Euphoria website down, members being lost

jeremy said...
useless said...

So why not a simple php/euphoria script that disallows more than a few requests a minute per ip, only non-duplicate requests, and only one request at a time?

This was done a few days ago. It's just a matter of not developing things you don't need. We never needed it, so it was never developed. Now that it is needed, it was developed and deployed.

Jeremy.

I was still getting a ton of 500 errors earlier this morning (approx 2 hours ago). I'd have to reload a page 5 times on average to be able to see it or to be able to submit a post.

new topic     » goto parent     » topic index » view message » categorize

20. Re: Euphoria website down, members being lost

CoJaBo said...

I notice that its still impossible to create an account, now due to coding errors.

./../euweb/edbi/edbi.e:392 in function last_insert_id()  
type_check failure, seq_name is 5  
... 

Also, valid email addresses are rejected.

Also, its still not possible to stay logged in when using a dynamic IP.

How, exactly, does Eu intend to grow if new members can't sign up?!

Hm, I just signed up fine as a new user. However, I will look at the reported problems today. I have just been way overloaded with real life this past week. Things are slowing down now so I'll work on it.

Also, I tried 10 valid email addresses, all were accepted. I tried a few invalid ones, they were not. Can you give me an example of valid email addresses that are being rejected?

Jeremy

new topic     » goto parent     » topic index » view message » categorize

21. Re: Euphoria website down, members being lost

jimcbrown said...

I was still getting a ton of 500 errors earlier this morning (approx 2 hours ago). I'd have to reload a page 5 times on average to be able to see it or to be able to submit a post.

The problem is that the ban is done in a Euphoria script, thus, the process is being launched to even check to see if the user is banned. I am going to work on using mod_rewrite and appending IPs who are hammering the system to just issue a Rejected response. Thus, we will not even launch a Euphoria process to determine if they should be banned or not, it will happen at the web server level.

Jeremy

new topic     » goto parent     » topic index » view message » categorize

22. Re: Euphoria website down, members being lost

jimcbrown said...
jeremy said...
useless said...

So why not a simple php/euphoria script that disallows more than a few requests a minute per ip, only non-duplicate requests, and only one request at a time?

This was done a few days ago. It's just a matter of not developing things you don't need. We never needed it, so it was never developed. Now that it is needed, it was developed and deployed.

Jeremy.

I was still getting a ton of 500 errors earlier this morning (approx 2 hours ago). I'd have to reload a page 5 times on average to be able to see it or to be able to submit a post.

Hm. I misunderstood. Static blocking was done a few days ago. Dynamic blocking is being written now.

Jeremy

new topic     » goto parent     » topic index » view message » categorize

23. Re: Euphoria website down, members being lost

TLDR

Proposed long-term solution:

  1. Build an eval() function, or equivalent "Euphoria-in-a-shell" shared library (DLL/SO) for dynamically running Euphoria code within a single process.
  2. Build a web server in Euphoria that uses multitasking to accept and process each requesting users's TCP connection.
  3. Use each task to process the appropriate web server activities, which includes launching Euphoria CGI apps with what was built in #1.
  4. ...?
  5. Profit!


-Greg

new topic     » goto parent     » topic index » view message » categorize

24. Re: Euphoria website down, members being lost

jeremy said...

Hm. I misunderstood. Static blocking was done a few days ago. Dynamic blocking is being written now.

OK, dynamic blocking is done. It's done in a matter that once euweb detects an attack, the htaccess file is updated to block the offending IP. This way, no processes are spawned when blocking the attack. The blocking is done 100% by the web server. We should be able to withstand tens of thousands of requests a second now. Currently the DOS attack is pretty small, only about 600 or so a second.

Jeremy

new topic     » goto parent     » topic index » view message » categorize

25. Re: Euphoria website down, members being lost

ghaberek said...

TLDR

Proposed long-term solution:

  1. Build an eval() function, or equivalent "Euphoria-in-a-shell" shared library (DLL/SO) for dynamically running Euphoria code within a single process.
  2. Build a web server in Euphoria that uses multitasking to accept and process each requesting users's TCP connection.
  3. Use each task to process the appropriate web server activities, which includes launching Euphoria CGI apps with what was built in #1.
  4. ...?
  5. Profit!

I have SCGI written in Euphoria. It is a single process that accepts requests via TCP/IP from a SCGI compliant server. It's pretty cool as the Euphoria process starts up, connects to the database, then sits idle waiting for a connection. Once it gets a connection, it processes the request and then starts waiting for another. This is a huge time saver as it does no parsing of Euphoria code for each request. It only does it once at start. It does not connect the database server each request, only each process that is run, which is once for every 5-6k requests.

My initial tests on a simple database app shows a CGI responding to about 80 requests a second. The same app run under a SCGI setting is nearing 1000 requests a second. The problem, again, is that our web host will not support such a process because it's really a daemon running, so, we are using CGI right now.

Jeremy

new topic     » goto parent     » topic index » view message » categorize

Search



Quick Links

User menu

Not signed in.

Misc Menu