OpenEuphoria

1. Experimental "peer to peer" modular programming concept

• Posted by ryanj May 06, 2015
• 1978 views

Working on large GUI applications over the years, i have always been frustrated with how difficult it can be to organize all the parts of an application, especially since there is usually special data processing mixed with user interface code and various states or views the application may switch to. Usually, i end up with a small main file and 2 or 3 huge files, with GUI and data processing code mixed together, and a few lower-level libraries that perform specific generic functions. It is difficult to define clear "modules" or some sort of hierarchy when different parts of the program need to communicate with other parts, so i end up with a make-shift message queue system that is difficult to maintain across different parts of 2 or 3 large files. Part of the problem is the way include files work assumes that an included file is somehow lower in the hierarchy, and there is no clear way to make different parts of the program "peers" or independent "modules" without some other library to include them and control them.

Does anyone else have this problem?

Possible solution: I have created an experimental "sync.e" library that allows any include file in a program to send messages without being aware of where they get received. Each file has to include "sync.e and subscribe to a "sync group", specifying a routine_id that will process sync messages. If any subscriber of the group calls sync:send(), all other subscribers' handler routines will be called by the sync.e library. Each library can then process incoming messages without a queue or tasks required.

No processing queue or controlling hierarchy is required. A main file can include many small .e files and not have to do anything with them once everything is initialized. Each .e file can be independent of the others, handling it's little part of the whole program. For example, if i create a file browser application, separate .e files could handle their own parts of a GUI: a folder tree, a location bar, a file list, etc. If any part of the gui is changed by the user, a message is sent out, and the other files process what they need to and update their part of the gui. Another file that is responsible for manipulating files could receive a message to do something with a specified file (open, delete, copy, move, rename, etc.) This makes the program very modular and easier to add or remove features later.

This is still experimental and theoretical, but i am currently rewriting all the source code of RedyCode to test this concept.

Here's the experimental sync.e library. Any thoughts? Let's discuss.

sequence 
gNames = {}, 
gSubscriberNames = {}, 
gSubscriberRids = {} 
 
atom debugRid = 0 
 
 
public procedure send(sequence groupname, sequence subscribername, sequence msgname, object msgdata) 
--sync data with other subscribers in group 
    atom idx = find(groupname, gNames) 
    if idx > 0 then 
        for s = 1 to length(gSubscriberNames[idx]) do 
            if not equal(gSubscriberNames[idx][s], subscribername) then 
                if gSubscriberRids[idx][s] > 0 then 
                    call_proc(gSubscriberRids[idx][s], {groupname, subscribername, msgname, msgdata}) 
                end if 
            end if  
        end for 
    end if 
    if debugRid > 0 then 
        call_proc(debugRid, {groupname, subscribername, msgname, msgdata}) 
    end if 
end procedure 
 
 
public procedure subscribe(sequence groupname, sequence subscribername, atom msghandlerid) 
--subscribe to a sync group to receive sync updates 
    atom idx = find(groupname, gNames) 
    if idx > 0 then 
        gSubscriberNames[idx] &= {subscribername} 
        gSubscriberRids[idx] &= {msghandlerid} 
    else 
        gNames &= {groupname} 
        gSubscriberNames &= {{subscribername}} 
        gSubscriberRids &= {{msghandlerid}} 
    end if 
end procedure 
 
 
public procedure unsubscribe(sequence groupname, sequence subscribername) 
--unsubscribe from a sync group 
    atom idx = find(groupname, gNames) 
    if idx > 0 then 
        gNames = remove(gNames, idx) 
        gSubscriberNames = remove(gSubscriberNames, idx) 
        gSubscriberRids = remove(gSubscriberRids, idx) 
    end if 
end procedure 
 
 
public function list_groups() 
--list all sync groups 
    return gNames 
end function 
 
 
public function list_subscribers(sequence groupname) 
--list all subscribers in a specified sync group 
    atom idx = find(groupname, gNames) 
    if idx > 0 then 
        return gSubscriberNames[idx] 
    else 
        return 0 
    end if 
end function 
 
 
public procedure debug(atom debughandlerid) 
--register a routine to be called for debug messages 
    debugRid = debughandlerid 
end procedure 

3. Re: Experimental "peer to peer" modular programming concept

• Posted by ryanj May 06, 2015
• 2001 views

Ah, i see. Interesting.

I like the "publish" terminology better. Thanks.

5. Re: Experimental "peer to peer" modular programming concept

• Posted by Ekhnat0n May 07, 2015
• 1929 views

This is a situation where a global variable of the struct-type will solve your problems.
By sending the result of any code producing results or part of them
to that struct and any code needing part of that result to retrieve it from.

7. Re: Experimental "peer to peer" modular programming concept

• Posted by Shian_Lee May 07, 2015
• 1912 views

I don't know RedyCode specifically, but in general, it's better to do the last heroic effort to move significant part of the code into generic library. Many times there is a way, although it's not obvious at all.

I wouldn't use a pattern (or tasks) unless I'm totally convinced that there is absolutely no way to create generic libraries for processing data, etc. In case that there is a way - a pattern might be not suitable.

There is nothing more modular then generic code, so it's worth a true effort.

9. Re: Experimental "peer to peer" modular programming concept

• Posted by ryanj May 08, 2015
• 1825 views

It is similar to GUI event handling, but more generic. Any part of the program can publish or subscribe to a topic. It is up to the programmer to define topics, message names, and message data formats, which should be listed in comments somewhere. I suppose this encourages good documentation and organization.

public procedure publish(sequence topicname, sequence subscribername, sequence msgname, object msgdata)  
--send data to other subscribers  to a topic by calling each subscriber's message handler proc 
    atom idx = find(topicname, gNames)  
    if idx > 0 then  
        for s = 1 to length(gSubscriberNames[idx]) do  
            if not equal(gSubscriberNames[idx][s], subscribername) then  
                if gSubscriberRids[idx][s] > 0 then  
                    call_proc(gSubscriberRids[idx][s], {topicname, subscribername, msgname, msgdata})  
                end if  
            end if   
        end for  
    end if  
    if debugRid > 0 then  
        call_proc(debugRid, {topicname, subscribername, msgname, msgdata})  
    end if  
end procedure 

11. Re: Experimental "peer to peer" modular programming concept

• Posted by Spock May 14, 2015
• 1787 views

Your situation (work + GUI) is probably similar to mine. Yet what I end up with in my programs is lots of include files each handling some low-level aspect. The problem is then compounded by having *additional* include files to manage the behaviour of groups of those files. The reason for this difference could be that I tend to refactor progams in order to compartmentalise unique/logical functionalities, eg, all vessels are listed in just one module, all customers are managed similarly in another module, all invoices are handled in another, etc.. the job of these modules is just to respond appropriately when interrogated by others. On top of that there are specific modules to link the GUI + behaviour aspects of the program.

I had started doing this refactoring with the GUI but got a bit sidetracked. The result would (will) have been having a separate module manage each control class. Does Redy have this structure already? I ain't looked recently.. One prime candidate for GUI refactoring is the list of constants. ATM I have a single list of many constants but very often a routine might only need access to a very specific group. I'd prefer to locate those constants inside the calling routine. But if those constants are ever needed elsewhere I have a problem. The solution could be a change to the language that might allow this:

function myfunc() 
   include constants.ew : mysmallgroup 
   .. 
end function 

In this theoretical case only the constants declared in mysmallgroup (which is a just a small section inside constants.ew) would be visible to the routine. Admittedly it is a bit revolutionary for Euphoria but it's the direction I would like to take a language.

The uni-directional behaviour of include could also be improved but mutually including modules are already possible in Euphoria (to my knowledge). What makes this a bit dangerous is how module initialising is done when there are dependencies. These days I tend to have a specific init() routine in each module that is called from a higher level after the GUI entry point.

Your idea of using publish-subscribe would certainly be useful in my programs, eg, when an event affecting many modules only needs to be transmitted as a message once rather than many times. Other than that I'm not sure it would necessarily improve other things. In general, I structure each module so that all handler routines are clustered at the end of the file and their job is solely to interpret the Windows messages as behaviours, ie, there is little or no actual work done by the message handlers - they simply pass that on to the main program.

Spock