Python has some language features that one might never use, but then
one day you come across them in some other code and need to know what
they do. The following fragments (see the reply by Xoanan on this page) are
To determine the intersection between two lists "list1" and "list2":
intersection = filter(lambda x:x in list1, list2)
To determine the union between two lists "list1" and "list2":
union = list1 + filter(lambda x:x not in list1, list2)
To determine the difference between two lists "list1" and "list2":
difference = filter(lambda x:x in list2, list1)
To determine the distinct elements, those not in common between two
lists "list1" and "list2":
distinct = filter(lambda x:x in not list2, list1) + filter(lambda x:x in not list1, list2)
A discussion of the various ways of extracting the
unique elements from a list
From time to time you might encounter a list which contains some lists,
and you want to flatten this into a single list of simple elements.
artical talks about two ways to do this. Perhaps the more readible
method is with the nested list comprehensions:
nested = [[1,2,3], [4,5], ]
flatList = [x for sub in nested for x in sub]
[1, 2, 3, 4, 5, 6]
this works by the first "for" loop (for
sub in nested) iterating over the top list and on each iteration
picking up a sublist and placing a reference to it in "sub", then the
second for loop (for x in sub)
runs and picks up the selected sub list and iterates over it, placing
each element of it in x. A list of all the individual values that x
takes is built up by the [x ...]
construct. A limitation with this is that all the elements in the
outer list must support iteration (i.e. be lists, tupples or something
else list-like), so you cannot have a simple scalar element in the
for Windows 2003 Server, the html
version from Google's cache. This talks a bit about additional
registry settings that might need applying to Windows to make a gigabit
LAN really perform well. This sort of tuning is also discussed here.
to consolize an arcade game, talks about the process of taking an
old commercial arcade game and extracting its guts to hook up to a TV
in your living room
22-Jul-03 Slashdot talks about the new Sony Vaio PCG-TR1A
sub-notebook which might make a pretty good webpad, but a few too many
bells and whistles make it a bit too expensive.
On 20-May-07 the OLPC
project got coverage on the US TV news journal show "60 Minutes",
while a lot of the focus was on the strife between OLPC and Intel,
Nicholas Negroponte was asked when Americans would be able to buy these
for their children, to which he replied that, this was planned for but
you would have to buy two: one for your child and one for a third world
child. Here's the quote from the
show: "If you’re wondering if the One Laptop will be available in
right now Negroponte’s in talks with some states and school districts.
He says it will be sold commercially in the future, but you’ll have to
buy two: one for your child and one for a child in a poor country."
- An introduction to using the Amazon EC2 system from Python. This also talks a bit about the actual technology behind the Amazon Elastic Computer Cloud (EC2). A video of the PyCon'08 talk on using the EC2 system to do cluster computing with Python. 
is trying to prevent "unauthorized links" to its site. Of course the only way
they can really enforce this is to remove their web site from the
internet... There is also some
information on related case law in the US where this idea was
tested on the basis of being a copyright infringement, but was thrown out
since the Judge found that it was "akin to a card cataglogue index system in
a library". Here is a similar
decison based on a lawsuit that Ford brought against 2600. This article
talks about the legality (US Law) of thumbnails and linking to images
on other sites.
- OpenID is a decentralized
identity system, it is discussed here
on Slashdot. In Feb'07 Bill Gates
announced that Microsoft will support it too. More discussion on OpenID, Yahoo, IBM, Microsoft, VeriSign and Google have all joined its board and there are now 250M OpenIDs in use. Here is a brief description of the process of getting an OpenID. MySpace has joined the OpenID coalition, adding a few more users. OpenID gets mentioned here in reference to attempts to move away from passwords to other means of authentication. Ned Batchelder found OpenID hard to get started in and dug up these discussions: OpenID is Why I Hate The Internet and The problem(s) with OpenID that talk about the difficulty of using OpenID and the apparent flaws in it. Microsoft has added support for OpenID to Windows Live, discussed here on Slashdot. OpenID for non-SuperUsers talks about setting up OpenID to use delegation. Not to be left out, Google is also supporting OpenID, but they have decided to fork development to address some of their concerns. More on Google's OpenID project here. Some sites are dropping support for OpenID. 
Bluetooth may not be as short ranged as previously thought,
talks about achieving 1km distances with it! Some of the security
concerns arrising are also mentioned.
- MSDN Magazine talks about extending the CLR for more support for dynamic languages like IronPython. 
- A webcast that talks about using Resolver (a cross between a spreadsheet and Python programming environment) to make dynamic, form-oriented, web pages. A write up of what Resolver does, and how it might be used. 
to safely add or replace a hard drive, has a good introduction to
IDE cables and jumpers and also mentions the trick of using Device Manager to uninstall the
current entries under Disk Drives
prior to imaging the system partition on the current old drive that is
to be reimaged to a new replacement drive. It also talks a bit about
the Windows Files and Settings
now that the 2GHz Pentium 4 is out Intel seems to have regained
the lead over AMD, according to this article
at least. Where the Athlon 1.4GHz seems to be lagging the most is in
CODEC type applications, which this article talks about a lot.
for Dummies, Steve Holzner, ISBN: 0-471-78597-0.
At about 350 pages this is not your typical massive web tome, but its
quite good and is fairly self-contained. It has a reasonable review of
the browser DOM and highlights the key points. It talks about XML,
initially this is kept brief enough to get you formatting the data to
be returned from the server and parsing it in the browser, later it is
discussed at some length. It spends about a quarter of the book
reviewing some of the available client and server frameworks, so if you
are thinking of taking that road it might be worth getting just for
Jan-Piet Mens talks about CAcert
- The HTTP1.0
draft discusses basic
authentication, and RFC2617 discusses digest authentication,
with some copies of the actual headers involved. RFC 1738 discusses
the syntax of Uniform Resource Locators (i.e. URLs). This article discusses
Another follow up to the issue
of logout. This article has a very good
discussion of the issues involved in implementing authentication.
Apparently this add-on
for Firefox adds the ability to logout (which is present in
versus Obscurity, a discussion on the use of salt. RFC 1867 talks about
the file upload ability that was added to html forms. RFC 2616 is a more
up to date HTTP specification.
Getting the spacing right can have an strong
influence on image sharpness when using a add-on converter lens.
This Gary's Parries article talks about this with the Raynox 6600 0.66x
wide converter used with the Canon A95.
Slashdot book review of: Linux
Programmer's Toolbox, by John Fusco, ISBN: 0132198576.
This talks about programming in the Linux environment.
This Slashdot article talks about some other systems
- Working with custom COM interfaces from Python, this talks about using the comtypes package to access COM objects with custom interfaces. 
- Bruce Schneier talks about the pros and cons of leaving a wireless LAN unsecured. This gets discussed here on Slashdot.  
- ACER is expected to enter the ultra portable laptop arena to compete with the ASUS Eee in the second quarter 2008. At CeBIT in Mar'08 they claimed to be on target for a Q2/Q3 release, though that's pretty vague sounding! ACER is claiming that its first 8.9 inch laptop will be priced in the $350-400 range, making it up to $150 less than the $500 ASUS is intending to charge. More information on this is leaking out. Pictures of the Aspire One appeared at the end of May-08. Engadget got their hands on one of these at Computex in June'08, talk is US$399 pricing and available in Sept'08. Acer has provided official specifications for this. Acer talks a bit about the role Linux is going to play in this market place and mentions a $379 version of the Aspire. The Tech Digest got their mitts on one. Laptop Mag reviews the Acer Aspire One. The Aspire One gets hacked to add an internal Bluetooth adapter. In late Aug'08 Acer announced price cuts to the Aspire One making it the lowest priced of the 9 inch display models (and even reduced the price of the model with the extended battery. 
- GIT (main
page here) is the revision control
system initially developed by Linus for the Linux kernel (manual page here), he
mentions it briefly in this article. A
talk by Linus on GIT is here,
in which he rants against CVS and SVN (some of which is quite well
deserved) and even Google. Currently it it requires Cygwin to run under
Windows, and as
such, it runs slower under Windows than on a Linux host. GIT gets
here along with a discussion of branch merging in Subversion. A brief
comparison of mercurial and git. Another comparison that
talks about how git's branching is more powerful than mercurial's. GitPython is a Python library that can work with Git repositories, so if you need to do something tedious or complicated there is a chance you could automate it with this and some Python code.
ways to write more comprehensible code talks about the joy of
- In Eating My Own Dogfood Christopher Lenz talks about his custom blogging software that he's writing in Python. 
- Samsung talks about flash reliability in SSD drives, they figure that due to the wear leveling technology a 100K write cycle flash will make it virtually impossible to wear out an SSD drive. For a rough approximation consider that your computer writes continually at a 1MB/s rate, then with an 32GB drive it would take 32K seconds to write once to all the cells. This would then need repeating 100K times, so its 32K x 100K or 3200M seconds, which is about 106 years. If you drop the drive size to only 4GB then you are still looking at 13 years (which is more than a mechanical drive is going to last). Increasing the write rate will also decrease the time, so if you bring it up to the maximum speed that such a drive can sustain, which is around 32MB/s then the ultimate life of a 32GB drive would drop by a factor of 32 to about 3.3 years. So you're not going to be able to wear out one of these drives within a 3 year warranty!
This sort of calculation also means that if a device like a compact flash drive is used in a computer as a system disk (so it's getting log files updated and the swap partition is on the drive) then so long as the device is large enough and the average write rate is acceptable then it will have a long life - and the easiest way to assure this is to just oversize the drive a bit. So instead of using a 512MB drive for your disk-less server, installing a 2GB unit will make it last 4 times as long. 
- Corsair's FAQ on flash drive wear leveling, this talks about both dynamic and static block wear leveling. 
- A preview of the new WorldWideTelescope that one of Microsoft's research labs has been working on. This project takes images from telescopes all over the world and stitches them together to produce a full view of the night sky that can be explored interactively. Discussed here on Slashdot. Now this will be a great way to use a home theater system driven by a good sized PC. This has now been launched. In March'09 a web interface to this (see it here) was added. 
- This lightning talk: Python Concurrency from PyCon 2008 by Jesse Noller gives an overview of the four major packages that are currently available for multi-processing from within Python. 
- This set of links to resources from the PyCon 2008 talks has a fair bit of information about distributed computing topics. 
- This site has a number of free maps, typically
in a black and white outline format, that can be useful for a variety
of purposes. The EPS format ones will load into Microsoft Publisher,
then you can pick a big page size and print a map that covers multiple
pages. The maps
at the CIA's World Factbook are of better quality, but they are in
PDF format which limits the tools you can use to manipulate them. This blog note talks about various sources for free vector maps of the world.
- The development of Windows
- Using Bazaar to hack on Twisted, talks about using a DVCS (distributed version control system) to coordinate work in an off-line environment for a project that is normally tracked by a conventional centralized version control system. The discussion here is for using Bazaar to work on a project in Subversion, but the same approach could be used with other DVCS systems like Mercurial. 
- On 8-Apr-08 Google started to preview their Google App Engine, discussed here on Slashdot.
Their overall design goals:
- make it easy to use
- make it easy to scale
- free to start (small apps)
What Google will do for you:
- Run the web applications
- provide the full life cycle, logs, status, updating, database...
- provide access to Google's scalable infrastructure, google accounts, big table, Google FS
To do this the application stack they provide has:
- Scalable serving infrastructure
- Python runtime
- Web based administration console
- Datastore (based on big Table)
Their environment does allow you to run a local test server, so you can do your application development on your own private machine.
They provide a basic Django template module.
Seems to follow the Python wsgiref module.
An initial presentation of this is in these videos.
One of the things this does is to get you to build things using Google tools which may result in an implementation that is difficult to move to some other service provider without doing a complete rewrite. Whereas if you were using Amazon's EC2 you are writing for a more standard LAMP style environment so you should be able to take whatever you develop and run it somewhere else. Of course, if you keep this all in mind it might not be a big issue, use the Google tools to develop a prototype and test the waters before investing in a full scale project.
With Google's use of Python as the first application language to be supported by this system it has caused an unprecedented stir in the Python community, see:
- The structure of .pyc files talks about what is in compiled Python bytecode files. 
- What is RSS talks about the name spaces that show up in RSS 1.0 feeds and has some Python code for parsing this. 
- Robust topological sorting and Tarjan's algorithm in Python talks about sorting graphs in the presence of cyclic links, this could be useful in things like garbage collection, dependency generation and revision control.  
- In App Engine and Pylons Ian Bicking talks about various Python modules that don't work on App Engine and why. He also talks about trying to monkeypatch around some of the issues. 
DVD players, also Nerd-Out
about the APEX-1500.
- pixie dust derived from the lining of a pig's bladder helped to regrow a finger tip. This may be a hoax, or strange luck but it's certainly the direction most people want this stuff to go. A talk about why we can't grow new body parts. 
- Ian Bicking talks about some of the various open source licenses, including the GPL and LGPL. 
- Switched On talks about the opportunity Linux has due to the growing market for low cost ultra-portable laptops started by the ASUS Eee.  
- A patent attorney talks about why we need to rethink intellectual property. 
- The MoneyTalks
- Caching RSS Feeds with feedcachetalks about using the feedcache package to better aggregate data from a number of RSS feeds in Python.  
- In Command line programs are classes, too! Doug Hellmann talks about using the CommandLineApp module to write command line applications that are classes. 
- A computer book addict talks about the computer book trade and mentions a rather large book store called Powell's (which is actually 5 stores) in Portland Oregon that might be worth a side trip. 
- Hunting memory leaks in Python talks about tracking down a memory leak caused by unintended global references keeping things alive (this sort of issue is really easy to hit in Java). This article uses the Graphviz utility to make a graph of all the objects currently being tracked by the Python garbage collector and then visual inspection can be used to find unexpected references that are causing problems. The author follows this up with another article that talks about how he used the gc and inspect modules to obtain the information from the garbage collector - a rather helpful discussion of this rather esoteric topic. Further discussion of object graphs using graphviz. Another problem is finding leaks in extension modules called from Python, for this something like Valgrind can be used. 
- The Zope Object DataBase (ZODB, the package is here) can be used without the rest of the Zope environment to give Python programs a transactional object database. A brief introduction to using it is here (and discusses a bit of how object references are maintained persistently. An IBM DeveloperWorks discussion of it with some simple use example code. The ZODB/ZEO Programming Guide is online here, a stand-alone PDF copy can be obtained here.
tempstorage provides a RAM based storage implementation for ZODB.
Introduction to the Zope Object Database provides a good summary of what the ZODB is, how it behaves and how to use it. FileStorageBackup talks about the design of the FileStorage type of storage system for ZODB, as well as the repozo tool for backing it up and some other integrity related tools. This article includes a list of tools that can inspect or analyze ZODB databases.
How to Love ZODB and Forget RDBMS.
CouchDB for ZODB Users takes a look at CouchDB and how it compares to ZODB. 
- Stroustrup talks about the origins of C++. Discussed here on Slashdot. His home page is here. 
- The Truth About Unicode in Python talks a bit about the limitations of UNICODE support in Python and some of the other things it might be desirable to have access to. 
- Mercurial Woes contains some observations about difficulties encountered when trying to use Mercurial. Mercurial Basics has some suggestions on how to avoid getting into trouble with Mercurial on a large project. Rebase is not the only way to deliver clean code talks about the popular but probably misguided use of rebase in distributed version control based projects. 
- The Iomega ScreenPlay TV Link is a small media player that does HDMI, component and composite video output at up to 1080i. The press release makes it sound like it might not actually play HiDef media as it talks about achieving 720p and 1080i output through upscaling. 
- Managing Meetings talks about using red, yellow and green cards to manage the flow of questions from the floor during a meeting. 
- In late Aug'08 more news about the Android phones started to appear, the Android Market application store and the winners of the Android Developer Challenge. 
- Baby steps with the Google App Engine talks about starting to deploy a web site under GAE. 
- Why your main program should be importable talks about a bit of Python style that is not obvious at first sight. 
- How to monitor drives or directories for changes in files on a Windows platform. This can be used to quickly identify files that need to get backed up, or published or new drives being attached that should be scanned. This example looks at listening for USB device changes. This article talks abut this in general terms and the first comment corrects things and makes mention of ReadDirectoryChangesW() and the need for watcher threads to respond to WM_DEVICECHANGE notifications so that if the user wants to disconnect a USB drive the thread can terminate. An old posting: CD Insert Notification in WinXP goes into some detail about handling the WM_DEVICECHANGED message in Python. 
- The Method Resolution Order (MRO) for Python 2.3 talks about the details of how the interpreter picks a particular method when multiple inheritance is present. 
- People like numbers that go up starts by talking about an idea for a game without numbers and then talks about how numbers often motivate people (and some of the problems this causes). 
- Ian Bicking talks about Decorators and Descriptors in Python. He suggests this article Introduction to Python Decorators as a good introduction. Decorators can be used to add arguments to functions. 
- An unusual approach to log file parsing talks about formatting log file messages as actual commands that can be executed. The result of this is that a log file might even be a full Python script that you run to analyze. 
- Cory Doctorow talks about Copyfight, the dynamic between culture and copyright. 
- Debugging Python regular expressions talks about using Kodos to do this. The comments also mention that the standard Python install includes
/Tools/scripts/redemo.py which provides a Tkinter GUI for doing this sort of thing. 
- On Remote Workers and Working Remotely talks about one person's experience with telecommuting. 
- Monitoring Hard Disks with SMART talks about using the smartctl utility from the smartmontools package that is included with many Linux distributions to monitor the health of hard drives. 
- How the Zope community does packaging with Grok. A reply to this talks about how pip requirements can be used to similar effect. 
- How to Photograph Mixed Light talks about taking photographs at dawn or dusk with a mixture of both natural and artificial light (which are at very dissimilar colour temperatures). This can be used to good effect in pictures of houses to create an inviting warm glow. The only real tricks to this are that there is only a relatively narrow window of time (about 10 minutes) and that you may need to manually set your camera's white balance to "sunny" (which will cause the blues of the sky and the yellows of the artificial light to get intensified). 
- Guido talks about the history of Python. 
- This paper: Secure Deletion of Data from Magnetic and Solid-State Memory talks about secure data deletion and has some epilogues that address recent changes in the field. 
- In My Manhattan Project: How I helped build the bomb that blew up Wall Street Michael Osinski talks about the software that was used to assemble sub-prime loans into tasty morsels that the market gorged on until it had a heart attack. Discussed here on Slashdot. 
- Pipe Fitting with Python Generators talks about using a pipe class along with generators and filters to examine a problem space. 
- The CSS Box Model talks about the various dimensions surrounding HTML elements as seen by CSS. 
- Of Catty Rants and Copyrights talks about the limits to republishing copyright material in a newspaper, in a strange case of a newspaper publishing a MySpace page as a letter to the editor without it having been submitted by the author. 
- In Trapped in Python Package; Send Food Jesse Noller talks about difficulties with Python package installations (primarily on OS/X and Fedora Linux) and how virtualenv can help address them. He also takes a look at building a python megapack package with all manner of useful things in it. 
- A Brief History of BitBacker, A Startup talks about the development of a on-line backup system that never made it to market.
- The singleton design pattern shows up in Python under the guise of modules, as modules are generally only imported once. This article talks about some of the difficulties with modules, especially when you need to reload them. 
- Some notes on using PySerial to talk to the serial port from Python. This uses com0com as a null-model emulator to demonstrate the communications. Another related article talks about how to list all serial ports on Windows with Python. A discussion on approaches to framing the data in serial communications, brings back memories of my old VBBS (IceBBS) days. A follow up that discusses frames and protocols. 
- A good article that talks about attacks by E-Mail and helps train users in how to recognize them. 
- Microsoft talks a bit about using online experimentation to test drive various web site changes to see how useful, desirable, profitable or costly they might be. 
- Python: Aggregating function arguments talks about the various ways to call functions and how to receive and process those arguments. 
- Engadget talks about using a wireless PMP for audio streaming. Of course that's going to be hell on its battery, and a number of commenters suggest that the better approach is to leave the streaming up to a PC and use the PMP to control it instead. Apparently the combination of iTunes and an iPhone or iTouch running remote control software will do the job well. 
- Python Logging 101 talks about using the logging module.  
- The ProFORMA software from Cambridge does 3D scanning using images from a stationary webcam. More details about this here and here on Slashdot. 
- I wonder why Apple never put an infrared receiver and transmitter in the iPhone/iTouch, as with it they would make very nice universal remote controls. You can "fix" this problem with the RedEye base which the iPhone talks to via WiFi and the base then sends the appropriate infrared controls to your home theater system. But still, its not a portable system and it costs nearly $200, to provide functionality that was standard in earlier PDAs like the Palm and probably costs about $0.25 to manufacture. An IR dongle type attachment (along with a universal remote application) has been announced, of course this probably makes more sense for use with the iTouch. The L5 iPhone IR dongle was shown at CES 2010. Logitech has released an app for the iPhone that turns an iPhone into a wireless trackpad or keyboard - this might make the iTouch a rather nice HTPC remote controller. The New Potato's FLPR IR dongle is another IR controller for about $80. The RedEye mini will bring the IR remote dongle down to about $49, it gets reviewed here. The i-Got-Control IRB1 is another entry into this market. 
- Building a DVD Server talks about putting together a storage and playback system for a DVD collection, unRAID (which I highly recommend) gets put to use as the storage server and software from Cinemar gets used to display the DVD catalog and control playback. 
- A presentation from the JVM Languages Summit 2009 that talks about how modern CPUs function. 
- Code reloading and event notifications talks a bit about how to dynamically reload code. 
- The Pattern-Seeking Fallacy talks about how we like to find patterns in things. 
- Gathering Leads using MTurk talks about using the Amazon Mechanical Turk project to gather data for a web site. 
- Saul's Simple Waxing System for Classic Cross Country Skis, in the first part he talks about sizing skis for proper glide and grip and reviews how temperature, time and humidity affect snow. In part 2 he discusses general properties of waxes. Part 3 talks about applying grip and klister waxes. Part 4 takes a look at a simple liquid glide wax.