ASPN : Python Cookbook : [Twisted] From block...
来源:百度文库 编辑:神马文学网 时间:2024/04/28 02:53:31
Title: [Twisted] From blocking functions to deferred functions
Submitter: Michele Simionato (other recipes)
Last Updated: 2005/08/19
Version no: 1.1
Category:Threads
--Rate-- 5 - Excellent 4 - Very good 3 - Good 2 - Needs work 1 - Poor
3 vote(s)
Description:
Twisted FAQs clearly state that "deferreds do not magically convert blocking
code into non-blocking code". So, how do you magically convert blocking
code into non-blocking code?
This recipe is the solution!
Source: Text Source
## version 1.1, changed according to the suggestions in the comments from twisted.internet import reactor, defer from twisted.python import threadable; threadable.init(1) import sys, time ## the wrong way def callInThread(func, *args): """Takes a blocking function an converts it into a deferred-valued function running in a separate thread. """ de = defer.Deferred() de.addCallback(func) reactor.callInThread(de.callback, *args) return de deferred = callInThread.__get__ # decorator associated to callInThread # the right way from twisted.internet.threads import deferToThread deferred = deferToThread.__get__ ## example code def print_(result): print result def running(): "Prints a few dots on stdout while the reactor is running." sys.stdout.write("."); sys.stdout.flush() reactor.callLater(.1, running) @deferred def sleep(sec): "A blocking function magically converted in a non-blocking one." print ‘start sleep %s‘ % sec time.sleep(sec) print ‘\nend sleep %s‘ % sec return "ok" if __name__ == "__main__": sleep(2).addBoth(print_) reactor.callLater(.1, running) reactor.callLater(3, reactor.stop) reactor.run()
Discussion:
How to make blocking code non-blocking is the obvious question for
everybody using Twisted, but the Twisted documentation does not make
easy to find the solution :-(
The trick is to run the blocking function in a separate thread. Here
all the magic is performed by the decorator, ``deferred``, which converts
``sleep``, a blocking function, into a deferred function i.e. a
non-blocking function that returns a deferred object. The code for
``callInThread`` is clear, and the ``.__get__`` trick converts
``callInThread`` in a one-argument function returning a closure,
i.e. an object suitable to be used as a decorator. I have seen this
trick in Alex Martelli‘s lectures at PyCon 2005. If you are confused
by it, you should read Raymond Hettinger essay on descriptors
(http://users.rcn.com/python/download/Descriptor.htm).
In short: every time you have a blocking function in your code, wrap
it with the ``deferred`` decorator and live happy!
Add comment
Number of comments: 2
deferToThread, Michele Simionato, 2005/08/18
Valentino Volonghi points out to me that there is already a function in Twisted doing the same job as my ‘callInThread‘, and doing it better: twisted.internet.threads.deferToThread. So just import it and define the decorator as ‘deferred=deferToThread.__get__‘ Valentino also points out that threads are not scalable, but I bet you already knew that ;)
Add comment
You don‘t want to use this code, Itamar Shtull-Trauring, 2005/08/18
Callbacks you add to the Deferred will run in the thread, but only *sometimes*. This makes thread safety really hard, possibly impossible and means lots of obscure hard to track down bugs. Instead, use twisted.internet.threads.deferToThread, where callbacks added to result will run in the Twisted main thread.
Add comment
Submitter: Michele Simionato (other recipes)
Last Updated: 2005/08/19
Version no: 1.1
Category:Threads
--Rate-- 5 - Excellent 4 - Very good 3 - Good 2 - Needs work 1 - Poor
3 vote(s)
Description:
Twisted FAQs clearly state that "deferreds do not magically convert blocking
code into non-blocking code". So, how do you magically convert blocking
code into non-blocking code?
This recipe is the solution!
Source: Text Source
## version 1.1, changed according to the suggestions in the comments from twisted.internet import reactor, defer from twisted.python import threadable; threadable.init(1) import sys, time ## the wrong way def callInThread(func, *args): """Takes a blocking function an converts it into a deferred-valued function running in a separate thread. """ de = defer.Deferred() de.addCallback(func) reactor.callInThread(de.callback, *args) return de deferred = callInThread.__get__ # decorator associated to callInThread # the right way from twisted.internet.threads import deferToThread deferred = deferToThread.__get__ ## example code def print_(result): print result def running(): "Prints a few dots on stdout while the reactor is running." sys.stdout.write("."); sys.stdout.flush() reactor.callLater(.1, running) @deferred def sleep(sec): "A blocking function magically converted in a non-blocking one." print ‘start sleep %s‘ % sec time.sleep(sec) print ‘\nend sleep %s‘ % sec return "ok" if __name__ == "__main__": sleep(2).addBoth(print_) reactor.callLater(.1, running) reactor.callLater(3, reactor.stop) reactor.run()
Discussion:
How to make blocking code non-blocking is the obvious question for
everybody using Twisted, but the Twisted documentation does not make
easy to find the solution :-(
The trick is to run the blocking function in a separate thread. Here
all the magic is performed by the decorator, ``deferred``, which converts
``sleep``, a blocking function, into a deferred function i.e. a
non-blocking function that returns a deferred object. The code for
``callInThread`` is clear, and the ``.__get__`` trick converts
``callInThread`` in a one-argument function returning a closure,
i.e. an object suitable to be used as a decorator. I have seen this
trick in Alex Martelli‘s lectures at PyCon 2005. If you are confused
by it, you should read Raymond Hettinger essay on descriptors
(http://users.rcn.com/python/download/Descriptor.htm).
In short: every time you have a blocking function in your code, wrap
it with the ``deferred`` decorator and live happy!
Add comment
Number of comments: 2
deferToThread, Michele Simionato, 2005/08/18
Valentino Volonghi points out to me that there is already a function in Twisted doing the same job as my ‘callInThread‘, and doing it better: twisted.internet.threads.deferToThread. So just import it and define the decorator as ‘deferred=deferToThread.__get__‘ Valentino also points out that threads are not scalable, but I bet you already knew that ;)
Add comment
You don‘t want to use this code, Itamar Shtull-Trauring, 2005/08/18
Callbacks you add to the Deferred will run in the thread, but only *sometimes*. This makes thread safety really hard, possibly impossible and means lots of obscure hard to track down bugs. Instead, use twisted.internet.threads.deferToThread, where callbacks added to result will run in the Twisted main thread.
Add comment
ASPN : Python Cookbook : [Twisted] From block...
Twisted
[Commits] python.it commit r205 - twisted/trunk/contrib/nevow/doc/txt
Connecting to ODBC Databases from Python with pyodbc
ARM 之 boot block
雷明顿Rolling block步枪
CookBook 菜谱配方软件界面设计
什么是Python?
Python 手册
Python学习
python twist
Python 是什么
python 多线程
Cell Phone SYSTEM BLOCK DIAGRAMS
Discovery could block blood clots
css中的block / inline 属性
C 博客 - Programmer‘s Cookbook - #pragma使用
A Byte of Python 程序语言-Python
Bloggers Block: Blog Tips at ProBlogger
偷窥Data block 的物理结构
Enterprise Library4.0 Cache Application Block...
Nicaraguan opposition vies to block court rul...
[天籁音乐] Nasri --Writer's Block
Python Cheat Sheet