1. Problem with FTP

I am developing a program with built-in sockets FTP and need some help.

When the program is performing a communication with the FTP server and the connection drops, the program would stay waiting for an answer, for a very long time, and all program elements (buttons, windows, menus) are inaccessible.

Trying to solve this, I installed a timer to perform a procedure that performs an ABORT command after 30 seconds but the timer is also locked. Please, anyone have any idea how to solve this problem?

Thank in advance.

Sérgio Gelli

new topic     » topic index » view message » categorize

2. Re: Problem with FTP

SergioGelli said...

I am developing a program with built-in sockets FTP and need some help.

Could you elaborate a little on this? Does this mean using the v4 std library?

Matt

new topic     » goto parent     » topic index » view message » categorize

3. Re: Problem with FTP

Here are a few suggestions (note: examples are based on eunet, from which Eu4 sockets are derived. Some of the procedure names may be a little different): 1) Poll the socket before each blocking send/recv call. Ex: status = eunet_poll(socket,POLLIN,100) 2) Use a non-blocking call instead. Ex: recvdata = eunet_recvfrom(socket,0) 3) If possible, figure out why the connection keeps dropping. The FTP program may be the lesser of several problem. 4) Modify the BLOCK_SIZE in sockets.e to be a smaller value. This will require more calls to recv for a given file, but you'll have more opportunities to check for a failed connection.

HTH, Michael J. Sabal

new topic     » goto parent     » topic index » view message » categorize

4. Re: Problem with FTP

Hi Matt and Michael

I am using "eunet.e" and "FTP stub" examples and V3.1.1.

The problem occurs when a line (as below) is executed and the connection to the Internt is very slow or dropped for some reason that does not matter here

eucode

status = eunet_send12(socketT,"NOOP "&13&10,0)

data = eunet_recv12(socketT,0)

end eucode

This is just one example, but the problem also occurs in all other commands (PASS, RETR, STOR and others)

The problem I have is that the program stops responding while waiting for response from the FTP, so I can not do any work on this program's window until it comes to server response (which eventually takes a long time)

new topic     » goto parent     » topic index » view message » categorize

5. Re: Problem with FTP

The forum didn't respect the newlines in my last reply, making it somewhat hard to read.

The main issue is that you are using blocking calls. Review the suggestions I made in my last post, and make sure you are using the non-blocking versions (recvfrom, sendto).

new topic     » goto parent     » topic index » view message » categorize

6. Re: Problem with FTP

o.k The "eunet_recvfrom (socketR, MSG_DONTWAIT)" works well. However, it is not very different from the "eunet_recv (socketR, 0)" In both cases, I can not do anything until the file is downloaded I want a function to stop receiving a file (or do anything else) without closing the program

Currently, the ways to stop the collection are: 1-Close the program 2-Burning PID 3-Wait Internet fall 4-Turn Off Computer

So, I still unresolved for my problem.

new topic     » goto parent     » topic index » view message » categorize

7. Re: Problem with FTP

sergelli said...

o.k The "eunet_recvfrom (socketR, MSG_DONTWAIT)" works well. However, it is not very different from the "eunet_recv (socketR, 0)" In both cases, I can not do anything until the file is downloaded I want a function to stop receiving a file (or do anything else) without closing the program

Currently, the ways to stop the collection are: 1-Close the program 2-Burning PID 3-Wait Internet fall 4-Turn Off Computer

So, I still unresolved for my problem.

If I understand correctly, the original problem (the GUI was frozen when the FTP connection dropped) is fixed, and now the GUI is responsive.

But now we have a new problem, in that although the GUI works, the program is still trying to finish downloading the file. And you want to change it so that when the FTP connection drops, the program immediately aborts the download?

new topic     » goto parent     » topic index » view message » categorize

8. Re: Problem with FTP

The default block size is 4K. This means, in order to receive a file larger than 4K, you have to be using a loop. If this is a command-line program, use get_key to check for a keyboard interrupt. If using win32lib, check the documentation for doEvents() and add a global flag that can be used to terminate the loop early. I'm afraid I can't help with wxWindows, but I'm sure there's a similar way to check or process the event queue from inside a loop. If you are trying to interrupt a file transfer of 4K or less, you will have to modify the BLOCK_SIZE constant in eunet.e to a smaller value.

new topic     » goto parent     » topic index » view message » categorize

9. Re: Problem with FTP

1-Yes jimcbrown, I want to change it so that when the FTP connection drops, the program immediately aborts the download, AND WHEN I USE THE PROGRAM TO OTHER THINGS WHILE I'M WAITING FOR AN END COMMUNICATION

2-Sabal Yes, I use a "while" for large files, because only then can receive several blocks of a file that is larger than my bockSize.

I understand that I can use (and am using) a "onEvent", between calls to a "eunet_recvfrom" or "eunet_recv", yet when the connection is too slow, I have to wait long for any part of the program return to work.

If I am not mistaken, some servers are configured to disconnect a connection that does not respond only after 900 seconds. Nobody wants to wait that long to use a program.

As I wrote earlier, I tried to use a "timer()" to have a chance to abort the operation from time to time, but the "timer()" is also frozen, while the program tries to communicate.

It seems that the ftp server is the owner of my computer :)

new topic     » goto parent     » topic index » view message » categorize

10. Re: Problem with FTP

SergioGelli said...

1-Yes jimcbrown, I want to change it so that when the FTP connection drops, the program immediately aborts the download, AND WHEN I USE THE PROGRAM TO OTHER THINGS WHILE I'M WAITING FOR AN END COMMUNICATION

I am confused as to what the problem is. Your wording suggests that you want toe FTP download to be aborted whenever you use the program to do anything else, which seems nonsensical to me.

I'm going to assume that

a) the FTP connection dropping continues to hang your GUI

b) the FTP download is not being aborted when the FTP connection fails

SergioGelli said...

2-Sabal Yes, I use a "while" for large files, because only then can receive several blocks of a file that is larger than my bockSize.

I understand that I can use (and am using) a "onEvent", between calls to a "eunet_recvfrom" or "eunet_recv", yet when the connection is too slow, I have to wait long for any part of the program return to work.

If I am not mistaken, some servers are configured to disconnect a connection that does not respond only after 900 seconds. Nobody wants to wait that long to use a program.

As I wrote earlier, I tried to use a "timer()" to have a chance to abort the operation from time to time, but the "timer()" is also frozen, while the program tries to communicate.

It seems that the ftp server is the owner of my computer :)

I think you need to do the following:

a) use eunet_recvfrom() with MSG_DONTWAIT

b) change the BLOCK_SIZE from 4K to a much smaller value

c) chuck out your while loop. if you are simply looping using eunet_recv() or eunet_recvfrom() until you've got the entire file, even if the socket is not blocking, the while loop will block your program until its finished. You need to rewrite this so that the data is only recv'd on an event call

Alternatively, you can stop using eunet for FTP and instead just use this:

system("start /b ftp "&ftp_url_to_download&", 2) 
new topic     » goto parent     » topic index » view message » categorize

11. Re: Problem with FTP

jimcbrown said...

a) the FTP connection dropping continues to hang your GUI b) the FTP download is not being aborted when the FTP connection fails

The connection need not fall, to hang the GUI.

Just the connection to be slow to hang the GUI.

Eg: See what happens when a server is unavailable:

1-The connection does not fall immediately. As I wrote earlier, some of them take 900 seconds to abort.

2-How the connection can be very slow or stop the looping "while" a very long time waiting for a new cycle. So I have to find another way to make the program work with other things, including the choice to abort a file download that was locked in "while"

jimcbrown said...

I think you need to do the following: a) use eunet_recvfrom() with MSG_DONTWAIT b) change the BLOCK_SIZE from 4K to a much smaller value c) chuck out your while loop. if you are simply looping using eunet_recv() or eunet_recvfrom() until you've got the entire file, even if the socket is not blocking, the while loop will block your program until its finished. You need to rewrite this so that the data is only recv'd on an event call

Alternatively, you can stop using eunet for FTP and instead just use this:

system("start /b ftp "&ftp_url_to_download&", 2) 

1-I am using eunet_recvfrom() with MSG_DONTWAIT.

2-If I change the BLOCK_SIZE from 4K to a much smaller value, like 256, and que server stop, my loop stop too..

3-system("start /b ftp "&ftp_url_to_download&", 2) This solution will stop responding exactly as described above, if the server stops responding or slow down, the advantage is the you cant type ctrl-c to abort...

new topic     » goto parent     » topic index » view message » categorize

12. Re: Problem with FTP

SergioGelli said...

3-system("start /b ftp "&ftp_url_to_download&", 2) This solution will stop responding exactly as described above, if the server stops responding or slow down, the advantage is the you cant type ctrl-c to abort...

That was not step three. It was an alternative to all three steps, and an alternative to using eunet.

Anyways, I do not understand why it has hung your GUI. start /b is *suppose* to run the command in the background - and return immediately to the calling program. So what is suppose to happen is that system() calls start, start runs ftp in the background, ftp.exe starts downloading the file while your program continues to operate (GUI fully functional and responsive).

SergioGelli said...
jimcbrown said...

a) the FTP connection dropping continues to hang your GUI b) the FTP download is not being aborted when the FTP connection fails

The connection need not fall, to hang the GUI.

Just the connection to be slow to hang the GUI.

Ok, so then you do have both problems as I describe below:

a) the FTP connection dropping *or being slow* continues to hang your GUI b) the FTP download is not being aborted when the FTP connection fails

SergioGelli said...
jimcbrown said...

I think you need to do the following: a) use eunet_recvfrom() with MSG_DONTWAIT b) change the BLOCK_SIZE from 4K to a much smaller value c) chuck out your while loop. if you are simply looping using eunet_recv() or eunet_recvfrom() until you've got the entire file, even if the socket is not blocking, the while loop will block your program until its finished. You need to rewrite this so that the data is only recv'd on an event call

1-I am using eunet_recvfrom() with MSG_DONTWAIT.

2-If I change the BLOCK_SIZE from 4K to a much smaller value, like 256, and que server stop, my loop stop too..

So it seems the problem is with step 3, then....

new topic     » goto parent     » topic index » view message » categorize

13. Re: Problem with FTP

system("start /b ftp "&ftp_url_to_download&", 2) The program can find out if the operation is over?

new topic     » goto parent     » topic index » view message » categorize

14. Re: Problem with FTP

SergioGelli said...

system("start /b ftp "&ftp_url_to_download&", 2) The program can find out if the operation is over?

No, not directly. If you know the size of the file being downloaded though then you can simply do a dir() on the file to find out if it is done yet or not.

Otherwise, the following batch script provides a work-around:

@echo off 
rem Lets assume this batch script is named ftpbatch.bat 
rem Usage: ftpbatch.bat ftp://site.tld/file/to/get c:\temp\ok_download_is_done.txt 
ftp "%1" 
echo %ERRORLEVEL% > "%2" 

and then you would do

system("start /b ftpbatch.bat "&ftp_url_to_download&" "&completion_file, 2) 

Then you could do a dir() on completion_file, and when it is created you know the ftp download is finished. (And if you need to see the exit code of ftp.exe to know if the download was successful or aborted, you can read the exit code in the file.)

new topic     » goto parent     » topic index » view message » categorize

15. Re: Problem with FTP

Thank you, your code looks very simple.

It is as the "egg of Columbus"

I do not know why I did not have this idea before! :)

But I have the program 98% done with "eunet.e" library So, I'll do some testing with your suggestions, but I will continue looking for a way to run the functions of "eunet.e" in the background. with the ability to intervene in the progress of tasks.

Any suggestions are still welcome

new topic     » goto parent     » topic index » view message » categorize

16. Re: Problem with FTP

In order to run any socket-based routine as a background operation, you need to have true threading. Euphoria does not currently support true threading (which with today's multi-core CPUs, isn't the best way to code anymore anyway), so the only way to do a parallel process is to spawn a separate program. Jim's suggestion is one way to do that, and is probably the simplest way of accomplishing your goals. This is one reason a full FTP implementation hasn't been included in eunet yet.

If you wanted a pure Euphoria solution, you could have your ftp routines built as a separate program, probably as a Windows service, and include either a file-based or socket-based inter-process communication API to allow your GUI to give instructions to the FTP process and query its status. Obviously that is a bit more complicated, but parallel programming usually is.

new topic     » goto parent     » topic index » view message » categorize

17. Re: Problem with FTP

m_sabal said...

you need to have true threading. Euphoria does not currently support true threading (which with today's multi-core CPUs, isn't the best way to code anymore anyway)

This is not true, and has not been true since the release of 4.0 beta3.

m_sabal said...

In order to run any socket-based routine as a background operation, you need to have true threading. Euphoria does not currently support true threading (which with today's multi-core CPUs, isn't the best way to code anymore anyway), so the only way to do a parallel process is to spawn a separate program. This is one reason a full FTP implementation hasn't been included in eunet yet.

If you wanted a pure Euphoria solution, you could have your ftp routines built as a separate program, probably as a Windows service, and include either a file-based or socket-based inter-process communication API to allow your GUI to give instructions to the FTP process and query its status. Obviously that is a bit more complicated, but parallel programming usually is.

I'm not sure if it is possible to build a service out of an Euphoria program. (You could always write a standard Eu program and install it through something like winserv, of course.)

new topic     » goto parent     » topic index » view message » categorize

18. Re: Problem with FTP

I think a small program Euphoria, using "eunet.e", for example, STOR.EXE, could be executed within the main program with "system("stor.exe parameter 1 parameter 2 parameter 3 ...", 2)" , this is like the suggestion made by JimcBrown.

This operation would be in the background, o.k.?

Meanwhile, the main program could monitor the completion of tasks and enable the User to choose an abortion, if it happens one undesired event.

But I do not know how to kill the execution of STOR.EXE, how can I do that? Does this will work?

new topic     » goto parent     » topic index » view message » categorize

19. Re: Problem with FTP

jimcbrown said...
m_sabal said...

you need to have true threading. Euphoria does not currently support true threading (which with today's multi-core CPUs, isn't the best way to code anymore anyway)

This is not true, and has not been true since the release of 4.0 beta3.

Sorry, I haven't been keeping up with the latest beta. So you've added full pthread support to the core, as opposed to the time-share algorithm we've had? If so, that's great. I still stand by my analysis, though, that threads are much less important and less useful today than they were a few years ago.

jimcbrown said...

I'm not sure if it is possible to build a service out of an Euphoria program. (You could always write a standard Eu program and install it through something like winserv, of course.)

I've been doing SaaS in Euphoria for a number of years now. See this contribution.

new topic     » goto parent     » topic index » view message » categorize

20. Re: Problem with FTP

m_sabal said...

you need to have true threading. Euphoria does not currently support true threading (which with today's multi-core CPUs, isn't the best way to code anymore anyway)

m_sabal said...

I still stand by my analysis, though, that threads are much less important and less useful today than they were a few years ago.

Huh? What analysis? Are you saying that pre-emptive multithreading is less useful than being able to take advantage of multi-core? Or are you saying that true multiprocess parallel programs are becoming more important these days?

m_sabal said...

I've been doing SaaS in Euphoria for a number of years now. See this contribution.

Cool. I'm impressed.

new topic     » goto parent     » topic index » view message » categorize

Search



Quick Links

User menu

Not signed in.

Misc Menu