Monday, September 30, 2013


TopCoder is an active programming community of developers who love to solve puzzles. There are many active challenges and some of them with cash prizes.
CodeKata is a blog of programming puzzles written by Dave Thomas, who’s most famous for the groundbreaking book, Pragmatic Programmer. The puzzles involve many issues that are directly relevant to real world programming so this is a good place to brush up on your coding skills.
Programming Praxis is a blog that includes a range of interesting problems with solutions usually available in several different programming languages.
Facebook has a collection of very challenging programming puzzles that–should you manage to solve them–could result in you getting a job at Facebook! Solutions are accepted in a variety of languages including Erlang, PHP, Perl, Python and Ruby.
Python Challenge features riddles that ask you to write small Python programs to solve. The difficulty level gets progressively harder and more cryptic. This is an excellent site for programmers/developers that like problem solving!
Quoted from their front page: “Project Euler is a series of challenging mathematical/computer programming problems that will require more than just mathematical insights to solve ” The puzzles featured on the site range from relatively trivial to seriously complex!
Ruby Quiz is a collection of Ruby programming challenges that is updated weekly. Although made for Ruby, these challenges can be solved in other languages.
Al Zimmermann’s self-described “arena where demented computer programmers compete for glory and for some cool prizes” is a great way to participate in the programming community. Contests run every six months.
Mind Cipher wasn’t included in the top 10 because this site doesn’t require you to do any programming. It does, however, include the “world’s greatest brain teasers, logical puzzles and mental challenges”, so if you’re just out for a quick mental workout this is the place to go!
99 Prolog Problems is suited to the Prolog programming language but people have provided solutions in Python, Ruby, Haskell, Scala and others. Working your way through these problems is a wonderful way to pick up a new language.
Virtualization delivers a host of benefits -- but that doesn't mean that everything is a good fit for a virtual environment. Here are 10 things that should probably stay physical. 
Virtualization provides a solid core of benefits -- cost savings, system consolidation, better use of resources, and improved administrative capabilities -- but it's important to remember that supporting the goals of the business are the reason IT departments exist in the first place. Virtualizing everything as far as the eye can see without analyzing the consequences is like what comedian Chris Rock said about driving a car with your feet: You can do it, but that doesn't make it a good idea.
The first step in any virtualization strategy should involve envisioning disaster recovery if you put all your eggs in the proverbial basket. Picture how you would need to proceed if your entire environment were down -- network devices, Active Directory domain controllers, email servers, etc. What if you've set up circular dependencies that will lock you out of your own systems? For instance, if you configure VMware's vCenter management server to depend on Active Directory for authentication, it will work fine so long as you have a domain controller available. But if your virtualized domain controller is powered off, that could be a problem. Of course, you can set up a local logon account for vCenter or split your domain controllers between virtual and physical systems, but the above situation represents a good example of how it might be possible to paint yourself into a corner.
In my experience, some things just aren't a good fit for a virtual environment. Here is my list of 10 things that should remain physical entities.

1: Anything with a dongle/required physical hardware

This one is a no-brainer, and it's been repeated countless times elsewhere, but -- like fire safety tips -- just because it may be a well-known mantra doesn't make it less significant. Believe it or not, some programs out there still require an attached piece of hardware (such as a dongle) to work. This piece of hardware is required by licensing for the program to work properly (to prevent piracy, for instance).
Case in point: An HVAC system for a client of mine ran on a creaking old desktop. The heating-and-cooling program required the use of a serial-attached dongle to administer the temperature, fans, etc. We tried valiantly to virtualize this system in a VMware ESXi 4.0 environment, using serial port pass through and even a USB adapter, but no luck. (I have heard this function may work in ESXi 5.) Ironically, this would have worked better using VMware workstation instead of the ESX environment, which did allow the pass-through functionality. But there was little point in hosting a VM on a PC, so we rebuilt the physical system and moved on.
This rule also applies to network devices like firewalls that use ASICs (application-specific integrated circuits) and switches that use GBICs (Gigabit interface converters). I have not found relevant information as to how these can be converted to a virtual environment. Even if you think you might cobble something together to get it to work, is it really worth the risk of downtime and administrative headaches, having a one-off setup like that?

2: Systems that require extreme performance

A computer or application that gobbles up RAM usage, disk I/Os, and CPU utilization (or requires multiple CPUs) may not be a good candidate for virtualization. Examples include video streaming, backup, database, and transaction processing systems. These are all physical boxes at my day job for this reason. Because a virtual program or machine runs in a "layer" on its host system, there will always be some level of performance sacrifice to the overhead involved, and the sacrifice likely tips the balance in favor of keeping it physical.
You might mitigate the issue by using a dedicated host with just the one program or server, but that detracts from the advantage of virtualization, which allows you to run many images on a dedicated physical server.

3: Applications/operating systems with license/support agreements that don't permit virtualization

This one is fairly self-explanatory. Check the license and support contract for anything before you virtualize it. You may find that you can't do that per the agreement, or if you proceed you'll be out of luck when it comes time to call support.
If it's a minor program that just prints out cubicle nameplates and the support agreement doesn't cover (or mention) virtualized versions, you might weigh the risk and proceed. If it's something mission critical, however, pay heed and leave it physical.
Which brings me to my next item…

4: Anything mission critical that hasn't been tested

You probably wouldn't be likely to take your mortgage payment to Las Vegas, put it down on at the roulette table, and then bet on black. For that matter, you definitely wouldn't gamble it all on number 7. The same goes for systems or services your company needs to stay afloat that you haven't tested on a virtual platform. Test first even if it takes time. Get a copy of the source (useSymantec Ghost or Acronis True Image to clone it if you can). Then, develop a testing plan and ensure that all aspects of the program or server work as expected. Do this during off-hours if needed. Believe me, finding problems at 11 PM on a Wednesday night is far preferable to 9 AM Thursday. Always leave the original source as is (merely shut it off, but don't disconnect/remove/uninstall) until you're sure the new destination works as you and your company anticipates. There's never a hurry when it comes to tying up loose ends.

5: Anything on which your physical environment depends

There are two points of failure for any virtual machine -- itself and its host. If you have software running on a VM that unlocks your office door when employees swipe their badges against a reader, that's going to allow them in only if both the VM and its parent system are healthy.
Picture arriving to work at 8 AM Monday to find a cluster of people outside the office door. "The badge reader isn't accepting our IDs!" they tell you. You deduce a system somewhere in the chain is down. Now what? Hope your master key isn't stored in a lockbox inside the data center or you'll have to call your security software vendor. Meanwhile, as employees depart for Dunkin' Donuts to let you sort out the mess, that lost labor will quickly pile up.
It may not just be security software and devices at stake here. I have a client with a highly evolved VMware environment utilizing clustering and SAN storage. And yet if they clone four virtual machines simultaneously, their virtualized Exchange 2010 Client Access Server will start jittering, even though it runs on another server with a separate disk (datastore). That server is being converted to a physical system to heal the issue. Yes, there is probably further tweaking and analysis that could be done to fix this, but in my client's view, solid Exchange connectivity is too valuable for them to experiment behind the scenes and hope for the best.

6: Anything on which your virtual environment depends

As I mentioned in the introduction, a circular dependency (such as a virtual domain controller being required to log into the virtual environment) puts you at a great risk once the inevitable downtime arrives -- and yes, even in clustered, redundant environments that day will come. Power is the big wildcard here, and if you live in the Northeast like me, I bet you've seen your share of power outages spike up just over the past five years.
I grouped this separately from the previous item because it requires a different way of thinking. Whereas you need to figure out the layout of your physical environment to keep the video cameras up and running, you need to map out your virtual environment, including the host systems, virtual images, authentication, network, storage, and even electrical connectivity. Take each item out of the mix and then figure out what the impact will be. Set up physically redundant systems (another domain controller, for instance) to cover your bases.

7: Anything that must be secured

This is a slightly different from rule #5. Any system containing secure information that you do not want other staff to access may be a security risk if virtualized. You can set up permissions on virtual machines to restrict others from being able to control them, but if those staff members have the ability to control the host systems your controls might be circumvented. They might still be able to copy the VMware files elsewhere, shut down the server, etc.
The point of this is not to say you should be suspicious of your IT staff, but there may be compliance guidelines or regulations that prohibit anyone other than your group from maintaining control of the programs/data/operating system involved.

8: Anything on which time sync is critical

Time synchronization works in a virtual environment -- for instance, VMware can sync time on a virtual machine with the host ESX server via the VMware tools application, and of course the operating systems themselves can be configured for time sync. But what if the operating systems forget or the host ESX server time is wrong? I observed this latter issue just a few weeks back. A set of virtual images had to have GMT for their processing software to work, but the ESX host time was incorrect, leading to a frustrating ordeal trying to figure out why the time on the virtual systems wouldn't stick properly.
This problem can be reined in by ensuring all physical hosts use NTP to standardize their clocks, but mistakes can still occur and settings can be lost or forgotten upon reboot. I've noticed this happening on several other occasions in the VMware ESX realm, such as after patching. If the system absolutely has to have to correct time, it may be better to keep it off the virtual stage.

9: Desktops that are running just fine

In the push for VDI (virtualized desktop infrastructure), some companies may get a bit overzealous in defining "what should be virtualized" as "anything that CAN be virtualized."  If you've got a fleet of PCs two or three years old, don't waste time converting them into VDI systems and replacing them with thin clients. There's no benefit or cost savings to that plan, and in fact it's a misuse of the benefits of virtualization.
It's a different story with older PCs that are sputtering along, or systems that are maxed out and need more juice under the hood. But otherwise, if it ain't broke, don't fix it.

10: Anything that is already a mess… or something sentimental

On more than one occasion I've seen a physical box transformed into a virtual machine so it can then be duplicated and preserved. In some situations, this has been helpful; but in others it has actually led to keeping an old cluttered operating system around far longer than it should have been. For example, a Windows XP machine already several years old was turned into a virtual image. As is, it had gone through numerous software updates, removals, readditions, etc. Fast forward a few more years (and MORE OS changes) and it's no surprise that now this XP system is experiencing strange issues with CPU overload and horrible response time. A new one is being built from scratch to replace it entirely. The better bet here would have been to create a brand new image from the start and install the necessary software in an orderly fashion, rather than bringing that banged-up OS online as a virtual system with all of its warts and blemishes.
The same goes for what I call "sentimental" systems. That label printing software that sits on an NT server and has been in your company for 15 years? Put it on an ice floe and wave good-bye. Don't be tempted to turn it into a virtual machine to keep it around just in case (I've found "just in case" can be the three most helpful and most detrimental words in IT) unless there is absolutely 0% chance of replacing it. However, if this is the case, don't forget to check rule #3!

Bonus: The physical machines hosting the virtual systems

I added this one in tongue-in-cheek, fashion, of course. It's intended to serve as a reminder that you must still plan to buy physical hardware and know your server specs, performance and storage needs, network connectivity, and other details to keep the servers -- and subsequently the virtual systems -- in tiptop shape. Make sure you're aware of the ramifications and differences between what the hosts need and what the images need, and keep researching and reviewing the latest updates from your virtualization providers.

Conclusion

As times change, these rules might change as well. Good documentation, training, and an in-depth understanding of your environment are crucial to planning the best balance of physical and virtual computing. Virtualization is a thing of beauty. But if a physical host goes down, the impact can be harsh -- and might even make you long for the days of "one physical server per function." As is always the case with any shiny new technology (cloud computing, for instance), figure out what makes sense for your company and its users and decide how you can best approach problems that can and will crop up.

Sunday, September 29, 2013

1. Network Penetration testing Guide: Risk assessment is a critical first-step in the information security lifecycle. Network penetration testing offers an invaluable way to establish a baseline assessment of security as it appears from outside the organization's network boundaries. (Requires free membership to view)

2. How to hack anything in Java: By Arshan Dabirsiaghi, Director of Research, Aspect Security. The white paper is about testing the security of Java applications which is considered practically more difficult than a similar browser-based client because inspecting, intercepting and altering application data is easy in the browser.

3. Mcafee on iPhone and iPad Security: This paper focuses specifically on helping security professionals understand the nuances of penetration testing iPhone/iPad applications. It attempts to cover the key steps the reader would need to understand such as setting up the test environment, installing the simulator, configuring the proxy tool and decompiling applications.

4. Engineering Principles for Information Technology Security: The purpose of the Engineering Principles for Information Technology (IT) Security (EP-ITS) is to present a list of system-level security principles to be considered in the design, development, and operation of an information system. 

5. A Good Collection of White papers on security and vulnerabilities: Not a white paper in itself but this site is a collection of white papers from different sources and some of these white papers are really worth referring. 
Securing DNS using DNS Forwarder Architecture
I. What is DNS Forwarder?
A DNS forwarder is a DNS in the network which used to forward queries related to public sites to external Internet DNS Servers.
II. Why DNS Forwarders?
1) Internal DNS Security
There are chances that your company users want to access a public website, which can't be resolved by the internal DNS Servers (which mainly handle records internal resources) in that case internal DNS servers will try to forward the query to extnernal DNS servers.
To provide extra security to DNS Servers, network administrators might not want Internal DNS servers to communicate directly with external Public DNS servers. We will implement a DNS server in DMZ network to communicate to external public DNS servers.Hence, you can resolve names without exposing your network to outside servers.
2) Less WAN Link Utilization
If all the Internal DNS servers start communicating to external Internet DNS servers ( which is via WAN link) the overall WAN link speed will go down. So we will just have a DMZ Network which will keep cahce of most queried extneral sites and reduce need for external queries.
III. DNS Forwarder Architecture
All Internal DNS Server should be set to forward query to Forwarder DNS Server, which is in our case hosted in DMZ Domain.
The Forwarder DNS Server should have good capacity to maintain cache to reduce external queries.
Forwarder DNS Server should be responsible for taking care of query until the query is resolved using help of External Internet DNS Servers.
(Refer to diagram to understand complete query process)
--
See you tomorrow with our new release! 
Programming is like a sport, the more you keep at it, the better you get. It takes a while to learn and you can not become a master programmer overnight. But, that does not mean that it is really tough either. All you need is a little patience.
Many have said that what is most important in order to be a master coder is a passion for computers. You have to really love computers in order to be patient enough. Being passionate is step number one. Here are 12 more steps that’ll get you started in your journey to being a code monkey. 

1. Getting started: Before you do, make sure that you really want to do this. Once you’ve done that, a big part of your job is done. This is the first step and the easiest one.
2. Do not be intimidated: Computers can surprise you at any point of time, so do not be intimidate when you suddenly realise that you don’t know even half of what you thought you did. It happens to the best of us.
3. Make a move: You can move to Silicon Valley, where almost everyone around you is a programmer and there are tons of more jobs. But, if you don’t have that option, make sure that you interact with programmers regularly.
4. Books: Buy the best programming books, this is an extension of the earlier step. Interact with programmers, find out the best books and buy them all. Read them too of course.
5. Find a teacher: Again, an extension of the interaction step. When all else fails, you should always have someone who you can look up to as the best programmer in the world!
6. Object Oriented: This is the programming language that revolutionised computers. Coders code in object oriented languages and eventually, they start speaking in such a language too. Read books on OOP and get it.
7. Make your code public: Again, programming forums are very important. When you write a piece of code, you should make it public for others to see. That is how you can find out how it could have been better or what you have done wrong. Sometimes, a program runs perfectly even when it is wrong or redundant.
8. Use github: Version control is very important in the world we live in. Learn to use Github and you’ll have added something to your arsenal.
9. Have your way: This might be pushing it, but programming is like art. Hence, you have to make what you think is good and what you like. Write a code for yourself, it drives you to do well.
10. Keep it simple: Now how the hell can a good code be simple right? It’s easy, a well written piece of code is always easy to understand. The classes etc. are well named, which makes them easier to understand. In addition, comments can be used wherever necessary to explain what your code is doing.
11. Branch out: Spread your wings and branch out. Make sure you write many different types of codes. That’s the secret to being a great coder. Eventually, you’ll realise that they are all very useful and related in many ways.
12. Linux, learn it: At least a basic grasp of Linux and the command line is needed in order to survive in the programming world. There is no way around it. Make sure you learn Linux the best you can.
Now, we know we said that this will get you started, but frankly, this is all there is to it. After this, it’s practice and hunger that’ll make you perfect.

1. Hacking Tutorials for Beginners - By BreakTheSecurity.com
The tutorial contains useful posts to help beginners understand the ethics of hacking. It also provides security tips on avoiding adware, windows firewall, online shopping, spyware prevention and so on.
2. How to learn Ethical hacking - By Astalavista.com
The site provides useful information on your devices, websites and assets. The site shows whether or not you’re hack proof.
3. Penetration Testing Tutorial - By Guru99.com
From here you can learn penetration testing, which is a type of security testing which helps you test the insecure areas of the system or application. It helps you learn Black Box Testing, White Box Penetration testing, and Grey Box Penetration Testing.
4. Backtrack Penetration Testing Tutorial
Backtrack Penetration Testing Tutorial is probably the best penetration testing distribution that provides some penetration testing programs that are used in this Backtrack Penetration Testing Tutorial.
5. Introduction to Penetration Testing
The current tutorial teaches you penetration testing, different types of vulnerabilities, security research, payloads, and so on. A useful resource for beginners!
6. Information Gathering with Nmap
The first tutorial to give you a basic walkthrough of a penetration testing! It focuses on information gathering, finding live hosts on the network, port scanning and versions of software that is running on those ports.
7. Simple How To Articles By Open Web Application Security
The tutorial contains numerous how to articles such as how to encrypt a properties file, how to meet verification reporting requirements, how to modify proxied conversations, how to perform a security architecture review at Level 1, how to perform a security architecture review at Level 2 and how to specify verification requirements in contracts, and so on.
8. The Six Dumbest Ideas in Computer Security
The tutorial describes what it considers six dumbest ideas in computer security starting from default permit, Enumerating Badness, Penetrate and Patch, Hacking is Cool, Educating Users, Action is Better Than Inaction , and so on.
9. Secure Design Principles
Understanding the basic principles puts you in a better position to execute particular practices where needed in your own projects. The principle of least privilege specifies that a user or computer program must be granted the least amount of privileges required to achieve a task. The tutorial describes all these principles for the users.
10. 10 steps to secure software The tutorial features some principles for developing secure software recommended by author and security analyst.
Hacking is definitely one subject loved by all, the adventure and the mystery hidden in hacking makes every geek feel curious about it. After bringing some interesting tutorials, we now bring to you some interesting hacking videos. We have shortlisted the long duration videos from YouTube… 

1. Open Security Training: In the spirit of OpenCourseWare and the Khan Academy, this YouTube channel is dedicated to sharing training material for computer security classes, on any topic, that are at least one day long.
All material is licensed with an open license like CreativeCommons, allowing anyone to use the material however they see fit, so long as they share modified works back to the community. This Youtube Playlist longs almost than 90 hours and is definitely the biggest video based tutorial ever.
2. OWASP AppSec USA 2011: YouTube Playlist containing compilation of OWASP conference highlight in 2011.
3. Defcon: How I Met your Girlfriend: Defcon is one of the most popular hackers conference. The presenters in this conference are well known in the hacking industry. This particular presentation will demonstrate the use of various open source tools for reversing Android Apps, as well as the use of the Android SDK features for pen testing, again including techniques and fast demos. Solutions to app and marketplace security will be covered as well.
4. Defcon: What happens when you steal a hackers computer: Having your place broken into and your computer stolen can be a nightmare. And getting your revenge from the thief who has your machine can be a dream come true. The presenter here experienced both of these when his machine was stolen in Boston and then showed up in Las Vegas 2 years later. He shares some laughs at a lamer’s expense, participate in the pwnage, and shows some resulting insights into the implications of certain security decisions.
5. Defcon: Nmap: Scanning the Internet: The Nmap Security Scanner was built to efficiently scan large networks, but Nmap’s author Fyodor has taken this to a new level by scanning millions of Internet hosts as part of the Worldscan project. He will present the most interesting findings and empirical statistics from these scans, along with practical advice for improving your own scan performance. Additional topics include detecting and subverting firewall and intrusion detection systems, dealing with quirky network configurations, and advanced host discovery and port scanning techniques. A quick overview of new Nmap features will also be provided.
6. Public Key Cryptography: Diffie-Hellman Key Exchange: Diffie-Hellman key exchange was one of the earliest practical implementations of key exchange within the field of cryptography. It relies on the discrete logarithm problem. This test clip will be part of the final chapter of Gambling with Secrets!
7. Web application Pen testing With Mutillidae: The Mutillidae project was introduced with the intention of using it as a teaching tool and making easy to understand video demos. They have been doing a lot of YouTube video tutorials are compiled on this page.
8. Intro to Scanning Nmap, Hping, Amap, TCPDump, Metasploit: This is the 2nd in a line of classes Jeremy Druin will be giving on pen-testing and web app security featuring Mutillidae for the Kentuckiana ISSA. This one covers scanning Nmap, Hping, Amap, TCPDump, Metasploit, etc.
We know how tricks and tips in any programming language are always an awesome thing to read for programmers out there! And what could be more fun to find some tricks in a language as complicated as Python. So while we were hunting for some Python support, we stumbled upon this interesting set of tricks shared by secnetix, and as sharing is the mantra of our community, here’s the list:

1. Emulating “?:”
Python doesn’t know the trinary operator “?:” from C. However, it’s pretty easy to emulate:
x ? y : z –> [z, y][bool(x)]
(If you’re sure that x is already a boolean type (or an integer of value 0 or 1), you can omit the bool() function, of course.)
How does the trick work? Simply create a small list constant, containing the two values to choose from, and use the boolean as an index into the list. “False” is equivalent to 0 and will select the first element, while “True” is equivalent to 1 and will select the second one.
Note that always all three operands will be evaluated, unlike the “?:” operator in C. If you need shortcut evaluation, use an if-else statement.
Actually, there’s another way to do it with shortcut evaluation, but it only works if y does not contain a boolean False equivalent:
x ? y : z –> bool(x) and y or z
2. Checking the Python version
You will often find yourself using features that are not available in older versions of Python. If someone happens to run your program with a too old version, an ugly exception traceback will be printed which is usually not very helpful to the user. Better use a snippet like this:
import sys
if not hasattr(sys, “hexversion”) or sys.hexversion < 0x020300f0:
sys.stderr.write(“Sorry, your Python is too old.\n”)
sys.stderr.write(“Please upgrade at least to 2.3.\n”)
sys.exit(1)
Those lines should be at the very top of your program, even before any other import statements. The variable hexversion is only available since Python 1.5.2, so we first check if it is there, just in case someone has an even older version. The format of the variable is 0x (major, minor and revision number, and an indication of the release status, which is f0 for official final releases). Each of the four parts is two hexadecimal digits.
3. Debugging CGIs
The “cgitb” module (available since Python 2.2) is extremely helpful when debugging CGI programs. Whenever a run-time error (i.e. exception) occurs, a nicely formatted HTML fragment will be produced, containing the backtrace with source context, line numbers and even contents of the variables involved. To use this module, put this line somewhere at the top of your CGI program:
import cgitb; cgitb.enable()
4. Parallel sorting of lists
Sometimes you want to sort a list, and there’s a second list (of the same length) which should be sorted along with it. Of course, you could have used a single list of 2-tuples in the first place, but sometimes a program requires a different structure. Anyway, it’s easy. The following snippet works with two or more lists (the example shows only two).
data = zip(list1, list2)
data.sort()
list1, list2 = map(lambda t: list(t), zip(*data))
Note that zip() returns a list of tuples, so you have to convert the result of the last zip() back to lists. That’s what the map() command along with the lambda function does. If you don’t actually need lists, the last line could be simplified like this:
tuple1, tuple2 = zip(*data)
5. Normalizing a MAC address
If you want to convert MAC addresses (sometimes called ethernet address) to a Canonical format, i.e. each of the six parts with two digits, and having all lowercase hexadecimal digits. It’s quite likely that there is an easier way, but this what you can do:
mac = “:”.join([i.zfill(2) for i in mac.split(":")]).lower()
(The zfill method of string objects is available since Python 2.2.2.)
6. Sorting IP addresses
How to sort a list of strings that represent IP addresses? Of course, you could supply an appropriate comparison function to the sort() method of the list object, but that’s very inefficient (read: slow).
It is better to first pre-process the list so it can be sorted with the efficient built-in comparison function (which simply compares strings character-by-character), and afterwards post-process the sorted list back to the normal format. That trick is applicable to a lot of sorting situations, not just IP addresses.
In the case of IP addresses, we re-format them so that each of the four octets is aligned inside a three-character field (preceded by spaces if necessary). Then all strings will be the same length and can be sorted using the fast built-in comparison function. Afterwards, we simply remove all spaces.
for i in range(len(ips)):
ips[i] = “%3s.%3s.%3s.%3s” % tuple(ips[i].split(“.”))
ips.sort()
for i in range(len(ips)):
ips[i] = ips[i].replace(” “, “”)
7. Parsing command line options
This is a code snippet for parsing command line options (using the getopt module) in a sophisticated way. In this example, the program accepts three options (both as one-letter options and as long options) and requires at least two arguments.
import sys, getopt, os.path
me = os.path.basename(sys.argv[0])
debug = False
really = True
verbose = False
my_options = (
(“d”, “debug”, “debug = True”, “Enable debug mode.”),
(“n”, “notreally”, “really = False”, “No action, display only.”),
(“v”, “verbose”, “verbose = True”, “Increase verbosity.”)
)
short_opts = reduce(lambda a, b: a + b[0], my_options, “”)
long_opts = map(lambda x: x[1], my_options)
def usage ():
args = “[-%s] [...]” % short_opts
print >> sys.stderr, “Usage: “, me, args, “\nOptions:”
for opt in my_options:
print >> sys.stderr, “-” + opt[0], opt[3]
sys.exit(1)
try:
opts, args = getopt.getopt(sys.argv[1:], short_opts, long_opts)
except getopt.GetoptError:
usage()
for o, p in opts:
for shrt, lng, action in my_options:
if o[1:] in shrt or o[2:] == lng:
exec action
break
else:
usage()
if len(args) < 2:
usage()
8. Mixing Python and shell scripts
Sometimes it might be useful to write a script that can be used as a shell script or as a Python script at the same time. This is possible. The trick is that a sequence of four quote characters means an empty string to the shell, but in Python it starts a triple-quoted string that begins with a quote character. So you can embed shell commands within that triple-quoted string. Note that the first string in a module or script is simply stored as the doc string for that module, but other than that it is simply ignored by the Python interpreter.
The following example demonstrates that trick. It is started as a shell script (because of the #!/bin/sh line). The embedded shell commands check if Python is installed. If not, a useful error message is displayed and the script exits. Otherwise, if Python is found, the script is re-executed with it. Python ignores the triple-quoted doc string and executes the rest like a normal Python program.
If you want to have a a real doc string for your program, you will have to assign it afterwards. The script below shows how to do this, too.
#!/bin/sh
“”"”:
if which python >/dev/null; then
exec python “$0″ “$@”
else
echo “${0##*/}: Python not found. Please install Python.” >&2
exit 1
fi
“”"
__doc__ = “”"
Demonstrate how to mix Python + shell script.
“”"
import sys
print “Hello World!”
print “This is Python”, sys.version
print “This is my argument vector:”, sys.argv
print “This is my doc string:”, __doc__
sys.exit (0)
Python is one of the most popular languages today amongst programmers. This is especially because Python is free to use, even for commercial products, thanks to its OSI-approved open source license. So when the language is open sources it will be highly unjustified to share paid books for Python. So here we bring to you 50 Free ebooks to help you understand, interpret and work with Python more smoothly and efficiently. This is absolutely in the spirit of open source! 

This cookbook is for experienced Python programmers who want to focus on modern tools and idioms. You’ll find complete recipes for more than a dozen topics, covering the core Python language as well as tasks common to a wide variety of applications.
Primary objective of this book is to explore the possibilities of using Python language as a tool for learning mathematics and science. The reader is not assumed to be familiar with computer programming. Ability to think logically is enough.
The book teaches complete beginners how to program in the Python programming language. The book features the source code to several ciphers and hacking programs for these ciphers. The programs include the Caesar cipher, transposition cipher, etc.
Effective Django development means building applications that are testable, maintainable, and scalable. After reading this book you should have an understanding of how Django’s pieces fit together and how to use them to engineer web applications.
The Python Game Book written by Horst Jens and friends aims to provide interested students and teachers a broad range of material to teach themselves how to write computer games using the language Python and other free Open Source tools.
Test-Driven Development with Python focuses on web development, with some coverage of JavaScript. This book uses a concrete example — the development of a website, from scratch — to teach the TDD methodology and how it applies to web programming.
PyQt4 is a toolkit for creating GUI applications. It is a blending of Python language and the successful Qt library. This tutorial is suited for beginners and intermediate programmers. You will learn to program non trivial PyQt4 applications.
This book describes Python, an open-source general-purpose interpreted programming language, available for all Platforms. Python is a high-level, structured programming language that can be used for a wide variety of programming tasks.
Teaching material on the scientific Python ecosystem, a quick introduction to central tools and techniques. The different chapters each correspond to a 1 to 2 hours course with increasing level of expertise, from beginner to expert.
Contents: Starting Out; Literals; Combining Literals; Precedence and Associativity; Variables; Assignment; Conditionals; Functions; Python Programs and Using Files; Input and Output; More about Functions; Scope; Loops; Lists; Recursion; etc.
Porting to Python 3 doesn’t have to be daunting. This book guides you through the process of porting your Python 2 code to Python 3. Using plenty of code examples is takes you cross the hurdles and shows you the new Python features.
Python is a widely used general purpose programming language, which happens to be well suited to Econometrics and other more general purpose data analysis tasks. These notes provide an introduction to Python for a beginning programmer.
With a primary focus on examples and applications of relevance to computational scientists, this useful book shows computational scientists how to develop tailored, flexible, and human-efficient working environments built from small scripts.
This book is a tutorial-style introduction to programming in Python for practicing historians (and other humanists). We assume that you’re starting out with no prior programming experience and only a basic understanding of computers.
The idea behind this book is to give an easily accessible entry point to hands-on computer vision with enough understanding of the underlying theory and algorithms to be a foundation for students, researchers and enthusiasts.
Contents: Starting Out; Literals; Combining Literals; Precedence and Associativity; Variables; Assignment; Conditionals; Functions; Python Programs and Using Files; Input and Output; Scope; Loops; Recursion; Arrays; Sorting; Footnotes; etc.
This book is about learning design patterns with Python language. If you are new to design patterns, this text provides the first building blocks. If you are interested in design of test automation frameworks, this book will be very useful.
This is a programming book that covers the Pygame game library for the Python programming language. Each chapter gives you the complete source code for a new game and teaches the important programming concepts from these examples.
The focus is on introducing programming techniques and developing good habits. Our approach avoids some more esoteric features of Python and concentrates on the programming basics that transfer directly to other imperative programming languages.
This book is an introduction to computer science using the Python programming language. It covers the basics of programming, including variables, functions, control flow, program debugging. Later chapters cover basic algorithms and data structures.
This book offers a highly accessible introduction to natural language processing, the field that supports a variety of language technologies. With it, you’ll learn how to write Python programs that work with large collections of unstructured text.
This is a very beginner book for people who want to learn to code. If you can already code then the book will probably drive you insane. It’s intended for people who have no coding chops to build up their skills before starting a more detailed book.
Think Stats is an introduction to Probability and Statistics for Python programmers. This new book emphasizes simple techniques you can use to explore real data sets and answer interesting statistical questions. Basic skills in Python are assumed.
The goal of this book is to provide an Informatics-oriented introduction to programming. The primary difference between a computer science approach and the Informatics approach taken in this book is a greater focus on using Python.
This book is meant to help you begin learning the basics of Python programming version 3 or later. It is a brief introduction to Python. The author attempts to be as brief as possible to get the new programmer into programming as fast as possible.
The goal of this book is twofold: to teach you how to program in Python; and to teach you to think like a computer scientist. This way of thinking combines some of the best features of mathematics, engineering, and natural science.
Guzdial introduces programming as a way of creating and manipulating media — a context familiar to today’s readers. Starts readers with actual programming early on. Puts programming in a relevant context (Computing for Communications).
Snake Wrangling for Kids is a printable electronic book, for children 8 years and older, who would like to learn computer programming. It covers the very basics of programming, and uses the Python 3 programming language to teach the concepts.
Django is the Python–based equivalent to the Ruby on Rails web development framework. The authors show you how they use this framework to create award–winning web sites. This is the first edition of the Django Book which covers 0.96 version.
This course is designed for biologists who already have some programming knowledge in other languages. The focus is on biological examples that are used throughout the course, as well as the suggested exercises drawn from the field of biology.
This document contains discussions of several advanced topics that are of interest to Python programmers: regular expressions, unit tests, extending and embedding Python, parsing, GUI applications, guidance on packages and modules.
This document is a syllabus for a first course in Python programming. It contains an introduction to the Python language, instruction in the important features of the language, and practical exercises in the use of those features.
This collection is a presentation of several small Python programs. They are aimed at intermediate programmers – people who have studied Python and are fairly comfortable with basic recursion and object oriented techniques.
The Python Imaging Library adds image processing capabilities to your Python interpreter. This library provides extensive file format support, an efficient internal representation, and fairly powerful image processing capabilities.
This tutorial introduces the reader informally to the basic concepts of the Python language and system. It introduces many of Python’s most noteworthy features, and will give you a good idea of the language’s flavor and style.
This reference covers invocation options, environment variables, lexical entities, basic types and their operations, advanced types, statements, iterators, generators, descriptors, decorators, built-in functions, built-in exceptions, and more.
The book will help you build basic programming skills. It is organized in a way that builds up the language in layers from simple concepts to more advanced features. Programming exercises are provided to encourage further exploration of each layer.
The definitive language reference for Python. It describes the syntax of Python and its built-in datatypes. It covers advanced topics, and is suitable for readers who are familiar with the details and rules of the Python and its object system.
The primary goal of this book is to promote object-oriented design using Python and to illustrate the use of the emerging object-oriented design patterns. The book shows how these patterns are used to create good software designs.
A concise introduction to software design using Python. Intended for people with no programming experience, this book starts with the most basic concepts and gradually adds new material. The goal is to teach you to think like a computer scientist.
Book for hobbyists, self-starters, and all who want to learn the art of computer programming in Python. Data types and variables, debugging, and namespaces are covered. Also includes sample applications that illustrate ideas and techniques in action.
This text teaches programming concepts to biologists. It is aimed at people who are not professional computer scientists, but who need a better control of computers for their own research. This course is part of a course in informatics for biology.
A hands-on tutorial that teaches how to accomplish numerous text processing tasks using Python. Filled with examples, the book provides efficient solutions to specific text processing problems and strategies for all text processing challenges.
This is a tutorial and a guide on Python programming for absolute beginners. If all you know about computers is how to save text files, then this is the book for you. The book is updated for the upcoming Python 3.0 language.
This free tutorial is designed to be an introduction to the Python programming language. The guide is for someone with no programming experience. The author attempts to teach programming by reading code and writing code.
This book is a complete presentation of the Python for professional programmers who need to learn the language. The author leads you from a tiny, easy to understand subset of statements to the entire Python language and all built-in data structures.
A large collection of useful Python scripts, the best parts of comp.lang.python newsgroup messages, plus hundreds of new scripts. The text covers the standard library, the examples should work on most platforms and Python versions.
This book covers application development using the library extension PyQt, which forms the basis for GUI programming. First part explains concepts using small examples, in the second part the author develops a complete, complex application.
This is a book for experienced programmers, a hands-on guide to the Python language. Each chapter starts with a complete code sample, picks it apart and explains the pieces, and then puts it all back together in a summary at the end.
This book is an introduction to Python, an easy to learn, powerful programming language. The tutorial explains the basics of the Python, it does not cover every single feature of the language, but introduces the most noteworthy features.