slopfront
Reddit, read-only. No JavaScript. No cookies. No tracking.
preferences

threading issue

6 points · submitted Mar 2016 by u/StorageNinja

I am putting together a threaded client/server program, which shows current throughput on the server screen.

The server receives the client connect and a thread is being spun up. That portion works just fine.

  • Client connect: Server says hello to client
  • The server starts a thread
  • The Client keeps on sending the the tx/rx info
  • Server shows the output on the screen

So far so good...

Here is the program, I want to keep track of the average for a specific period of time.

the class for the client:

class client(object):
    def __init__(self, name):
        self.name=name
        self.txtotal=0
        self.rxtotal=0
    def txrxinc(self, tx, rx):
        self.txtotal+=tx
        self.rxtotal+=rx
    def print_average(self):
        while True:
            time.sleep(AVERAGE)
            lock.acquire(True)
            rx_bytes, rx_kbytes, rx_mbytes=converter(self.rxtotal/AVERAGE)
            tx_bytes, tx_kbytes, tx_mbytes=converter(self.txtotal/AVERAGE)
            print socket.gethostbyaddr(clientaddress[0])[0]+"("+clientaddress[0]+")", "Average:", rx_bytes, rx_kbytes, rx_mbytes, tx_bytes, tx_kbytes, tx_mbytes
            self.txtotal=0
            self.rxtotal=0
            lock.release()

As you can see, "while True:" keeps on looping as its own thread. Every time the time.sleep(AVERAGE) hits, it is supposed to print out the average on the screen.

Now, the above function gets called like this. I will only show the first couple of lines, because it just hangs with "while True:" in place

def replyback(clientsocket, clientaddress):
    CLIENTS[clientaddress[0]]=client(clientaddress[0])
    ThroughPutThread=threading.Thread(target=CLIENTS[clientaddress[0]].print_average())
    ThroughPutThread.daemon=True
    ThroughPutThread.start()
    print CLIENTS

"print CLIENTS" gets never started, if the "while True:" loop is active. If I take "while True:" out, we only run "print_average" once and the server keeps on getting all the client rx/tx info.

In short, If I disable the loop, everything works.

I am not sure what I am doing wrong.... It's like it is not running as a thread.

12 comments · 100% upvoted · other discussions

comments

JohnnyJordaan 2 points · Mar 2016

Can you explain why you are running threads this way? Why not consider a client object a thread by itself?

class MyClient(threading.Thread):
    def __init__(self, name):
        super(MyClient, self).__init__()
        self.name=name
        self.daemon=True
        self.txtotal=0
        self.rxtotal=0
    def txrxinc(self, tx, rx):
        self.txtotal+=tx
        self.rxtotal+=rx
    def run(self):
        while True:
            time.sleep(AVERAGE)
            with lock:
                rx_bytes, rx_kbytes, rx_mbytes=converter(self.rxtotal/AVERAGE)
                tx_bytes, tx_kbytes, tx_mbytes=converter(self.txtotal/AVERAGE)
                print socket.gethostbyaddr(clientaddress[0])[0]+"("+clientaddress[0]+")", "Average:", rx_bytes, rx_kbytes, rx_mbytes, tx_bytes, tx_kbytes, tx_mbytes
                self.txtotal=0
                self.rxtotal=0

def replyback(clientsocket, clientaddress):
    ThroughPutThread=Client(clientaddress[0])
    ThroughPutThread.start()
    CLIENTS[clientaddress[0]] = ThroughPutThread
    print CLIENTS

I'm still wondering what the context of all this is, so if you could share your full code we can give more feedback I think.

StorageNinja [OP] 1 points · Mar 2016

I need two running threads per client... unless someone has a better implementation idea.

The initial connection thread is the client connect, who gives me his tx/rx every second and shows it scrolling on the server. This will loop forever until you kill either the client or server. The second thread created from the first thread, is the average tx/rx.

example:

  • some.client.com - thread a - tx/rx
  • some.client.com - thread a - tx/rx
  • some.client.com - thread a - tx/rx
  • some.client.com - thread a - tx/rx
  • some.client.com - thread a - tx/rx
  • AVERAGE/second some.client.com - thread b - tx/rx
  • some.client.com - thread a - tx/rx
  • some.client.com - thread a - tx/rx
  • some.client.com - thread a - tx/rx
  • some.client.com - thread a - tx/rx
  • some.client.com - thread a - tx/rx
  • ....
zifnabxar 1 points · Mar 2016

You should be able to compute the average and the per second with one thread. It has to compute a little more, but it should also be a little easier on the parallelism and threading.

StorageNinja [OP] 1 points · Mar 2016

oohhhhh... I think i know what you are getting at, but would that really make a difference? Does it make a difference, if just a method or the entire object a thread is?

JohnnyJordaan 1 points · Mar 2016

Not per se, but calling an object's method as a Thread is kinda strange. My experience is that if you keep your code intuitive and clear, you won't run into many mysterious behaviours like this one.

Maybe it won't fix the problem directly but at least it's a good start.

StorageNinja [OP] 1 points · Mar 2016

I had some time available to get back to this today. Your idea was great! The solution was to make the class a thread instead of the method within the class.

Lesson learned: do not make a method within a class a thread, but rather the entire class.

http://imgur.com/q8HMwkt

zifnabxar 1 points · Mar 2016

It looks like the client thread is blocked on the lock.acquire() call. How are you creating the lock? Is anyone else acquiring it? When/how do they release it?

It would also help to know the version of Python and your OS. Python threading/multiprocessing has some annoying quirks.

zifnabxar 1 points · Mar 2016

Quick follow-up. Do you know what I mean when I say "is blocked on the lock.acquire() call"? Judging from what you're doing and how you've explained yourself, I assume that's the case, but I want to make sure.

StorageNinja [OP] 1 points · Mar 2016

Initially, I thought the "lock" may be causing this. For testing purposes, I kicked it out, but it was still behaving the same. Currently, I am using only one client for my testing. This is the only "lock" statement. The "lock" is just around the print statements, which will print out the average.

I am writing the script in a lab env. on linux / python 2.7.11.

I may have found something, but this makes this even stranger. For the threading starting procedure, I added "name=".

Server start:

while True:
    clientsocket, clientaddress=serversocket.accept()
    print "Connection from", socket.gethostbyaddr(clientaddress[0])[0]+"("+clientaddress[0]+")", "on port", str(clientaddress[1])+"."
    t=Thread(name="connection thread "+socket.gethostbyaddr(clientaddress[0])[0], target=replyback, args=[clientsocket, clientaddress])
    t.daemon=True
    t.start()

Pretty simple, we keep on looping waiting for connections. As soon as we get a connection, we create a thread and assign it the name "connection thread some.server.com"

When it calls replyback, I added a print out of the thread:

def replyback(clientsocket, clientaddress):
    print "reply back: ", current_thread().name
    CLIENTS[clientaddress[0]]=client(clientaddress[0])
    ThroughPutThread=Thread(name="Through Put thread"+clientaddress[0],target=CLIENTS[clientaddress[0]].print_average())
    ThroughPutThread.daemon=True
    ThroughPutThread.start()

This prints out "connection thread some.server.com" Awesome!

The next thread creation starts with added name as well:

ThroughPutThread=Thread(name="Through Put thread"+clientaddress[0],target=CLIENTS[clientaddress[0]].print_average())

Now, if it calls object print_average(), I added another

print "print average: ",current_thread().name

I would expect a print out like this "Through Put thread some.server.com". The thing is... it replies back "connection from thread some.server.com"

This would explain why this is not running as a new thread.

The new "thread" runs under the initial thread id!

Right? follow up question? How to fix this? lol

zifnabxar 1 points · Mar 2016

What do you mean by "the next thread creation"? Clearly client isn't being called, but it's hard for me to understand why without seeing more of your code.

Is there any chance I could see all of your code? Either a github repo or a pastebin? Threading/multiprocessing is difficult to grok

StorageNinja [OP] 1 points · Mar 2016

http://pastebin.com/9P0A2mCk

zifnabxar 1 points · Mar 2016

If I run your code and then connect to it with my browser, I get what I believe is the correct response:

-> Create socket object
-> using port 55555
-> bind to rosalind on port 55555
-> waiting for up to 10 clients
-> getting an average every 10.0 seconds
Connection from localhost(127.0.0.1) on port 49380.
reply back:  connection thread localhost
print average:  connection thread localhost

print_average is being called. Is this not what you're seeing?