Joe Zawodny

Wavelet Processing Screen

As promised, I coded up an add-on to AviStack that performs wavelet based image enhancement. The functionality is similar to the Wavelets found in RegiStax, but are subtely different as you’ll see in a moment. But before I tell you more about the new capability coming to AviStack, I’d like to thank Michael Theusner, author of AviStack, for giving me the opportunity to contribute, and Cor Berrevoets, author of RegiStax, for discussing the Wavelet implementation in RegiStax with me. This wavelet processing capability should appear in V1.74 of AviStack.

There are a lot of different ways to define and work with wavelets. Wavelets are most commonly used in image compression. For the image enhancement process, we use something called a stationary wavelet transform. Basically, this is a special case of the wavelet transform where the image is not decimated between levels in the filter bank. A filter bank is a way of describing how an image is decomposed through the application of smoothing functions called kernels. The original image is smoothed by convolution with a kernel and the difference between the original and smoothed version is called a layer 1. The smoothed version is then convolved with another broader kernel. The difference between those two are called layer 2. This iterative process is repeated for every kernel/layer in the filter bank.

For image enhancement, the kernels are selected to capture or represent detail in the image at certain resolution scales. This module uses a Gaussian kernel and the resolution scale is represented by the Sigma value (sigma * 2.35 = FWHM for a Gaussian). Each layer has a Sigma as well as an Amplitude. Normally, if you take the last smoothed image from the filter bank and add in all of the layers you get the original image. If instead, you multiply a layer by a factor other than 1, you can enhance or suppress the detail associated with that layer. The Amplitude value in each filter/layer is just the multiplicative factor (well almost as you’ll see). At least that is the way it works in RegiStax. In my version for AviStack, I decided to streamline the code significantly, by adding the Layer times the Amplitude to the original image. So with AviStack, setting all of the amplitudes to zero gives the original image. It is a subtle but important difference that you’ll need to remember if you are a regular RegiStax user. Based on a quick survey of RegiStax users, I decided to only implement a Gaussian kernel. I could not find anyone who uses the default. One thing that worried me while enhancing images in RegiStax was that I could easily blow out the highlights. So for AviStack, I added a Show Clipping button which shows the user which pixels will be clipped to white if the image is saved as a 16-bit format. Lastly, I added a control to adjust the image gamma. I think you will find that this wavelet processing is very fast compared to RegiStax, so the results of any changes you make are immediately displayed. I’m not sure whether most RegiStax users fully understood the implications of the Initial and Step controls, so I did not implement them. The user is expected to select the appropriate Sigma value - which in almost all cases should increase with successive layers (see figure as an example). Let’s get on with an introduction on how to use wavelets in AviStack.

Usage:
There are six sets of controls for the six wavelet levels. Each set is comprised of a Sigma and Amplitude control. The Sigma control sets the width of the Gaussian for that wavelet. The current valid range of Sigma is from 0.16 to 2.00. The Amplitude control determines how strongly the wavelet is applied to the image. Its range is from -20 to 200. Clicking on any slider or Clicking & Dragging the slider bar will allow you to change the value. For single step adjustment (0.01), arrows are provided at each end. Results are displayed for the entire image immediately (or after the button release of a Drag). Near the edge of the image, data required for the wavelet does not exist. Rather than create data, the wavelet is not applied and instead the image is cropped. The number of pixels cropped from each side can be estimated by the following formula (#pixels = Integer( (largest sigma) / 0.16666)). So, if the largest sigma is 0.3 then 1 pixels will be cropped from each side. Two will be cropped if the largest sigma is 0.34 - and so on.

Pressing and holding the Show Clipping button below the wavelet controls will show where the processed image goes beyond the 16-bit range of TIFF or PNG files. Bright pixels with values beyond 65535 display as black while this button is held down. It is useful for monitoring the effect of wavelet processing on the detail in the brightest areas of the image.

There is a control to adjust the gamma. Again, results are displayed immediately.

There are a set of save buttons for FIT, 16-bit TIFF and 16-bit PNG files. Selecting one of these will bring up the usual save dialog. The file name defaults to the name of the original file with ‘_wvs’ appended followed by the appropriate suffix for the file type.

Finally, the two remaining buttons complete wavelet processing and return the use to the main AviStack screen. These currently do exactly the same thing - exit the wavelet screen, but will behave differently in a future version of AviStack.

Differences from RegiStax:
While these controls should be familiar to a RegiStax user, there are differences. The Sigma value is roughly equivalent to the RegiStax value times 2. Unlike RegiStax, the Sigma should get larger with successive layers since there is no other scaling being applied to the width. The Amplitude setting is a multiplicative factor applied to the wavelet. Unlike RegiStax where an amplitude of 1 represents no change, the enhancement of wavelet layers in AviStack are effectively turned off of by setting the amplitude to 0. Any zero amplitude layer followed by a non-zero amplitude layer is still calculated and impacts subsequent wavelet layers. Only those wavelets that are needed are applied. For example, If the third wavelet is the last non-zero amplitude layer, then only 3 layers of wavelets are used (as seen in the figure). Negative amplitude combined with larger sigma in the last layer used has the effect of reducing the degree of clipping in the brightest areas of the image.

Well as long as I’m taking shots here today. Caroline Kennedy is, “ya know”, not exactly qualified for the Senate - by any stretch of the imagination. Simply being a Kennedy is insufficient qualification. She’s been, for all practical purposes, in hiding for decades and now pops up (with someone) wanting to make certain that there is a Kennedy in the Senate. There must be several other truly qualified and established public servants in NY that should be higher on any one’s list of potential appointees. Is this simply an ongoing media love affair with the Kennedy name or something more behind the scenes?

Oh (now for a cheap shot), and what is with that nose/spoiler/winglet thing she has going on? Has she been trying out for a part as an alien in Star Trek (Worf’s long lost half sister or something)? It certainly, “ya know”, looked better on her mother. If she makes it to the Senate at least the cartoonists will have a bizarre facial feature to grotesquely caricaturize.

I’m going to cross a few lines here. If you disagree, please feel free to comment. I’m open minded and can be persuaded to change my point of view. Having said that, here I go.

First some context, I think it was arrogant and short-sighted when Israel was created after WWII. The local population did not see the Empire as having the authority to redraw the map and give away their land. I believe this act is the root of the current hostility. I can imagine other things contributing, but this is certainly the main problem. If this is true, then there can only be two long lasting solutions to the problem.

There have been so many attempts to solve the Middle East problem that there isn’t enough room or time to summarize them in a meaningful way. Let me just say that there has been a lot of give and take, but there has never been an agreement where every one of the parties that needed to partake could fully support. There are now so many factions in the area that is it is virtually impossible to get complete agreement.

These agreements and cease fires are tenuous at best partly because of the eye-for-an-eye Israeli way of neighborly foreign relations. All it seems to take is for a small group of neredowells to lob a few rockets over the border and we are where we are today - AGAIN. Honestly, I really can’t blame them. I have a hard enough time keeping my neighbors from driving on my lawn as they back out of their driveways. It is a good thing I do not own nukes, but I digress. The Israelis are not without blame. They do have a tendency to overuse bulldozers and erect settlements in sensitive areas with a thumbing-their-nose attitude.

I do not have a solution to the problem. I’m not that smart. Well I do have a solution, but it is a certainly too heavy handed. There are surely too many weapons in the area and they are in the hands of too many autonomous factions whose power is derived from sustained hostilities. If the neighboring governments had any control and authority over their countries and populace, there might be a ray of hope for peace in the region. As it is now, it reminds me of our Wild West days. I would not be surprised if things get very very heavy handed in the coming weeks.

I have a hard time decoding the oft quoted ‘we have a Special Relationship with Israel’ concept. I do believe that we have a fundamental responsibility for creating the problem in the first place. Honestly, if all of this was happening in the middle of Africa, like so many other and more horrible things are occurring today, we really would not care so much (not that we should not care). If it were not for the potential disruption of the flow of oil from the region and it, therefore, became just a moral issue as opposed to an economic issue, I do not think we would have a ‘Special Relationship’.

So other than making me feel a little better, where does this leave things? Although we would like to think of ourselves and the world as civilized, we are far far from it. We are too obsessed with Our Needs, Our Desires, and imposing Our Way on everyone else. We appear incapable of caring about anyone else, especially if they are not just like us. Civilized - yeah right!

Another year has just about passed and it is always a good time to look back and reflect. A lot has happened this year, some good and some bad. For us it has been good enough in that we are all still here and in reasonably good health and spirits. We hope the same is true for you and wishing that the next year will be better than the last.

Merry Christmas and Happy New Year!

I was just reading this story on CNN and began thinking (always something to be worried about). In the old days, deflation was a worry because the reduction in prices meant cut backs in production which led to job losses and salary decreases which then reinforced the deflationary cycle. Those were the old days. Now we hardly produce anything here - in the US - anymore. The production is overseas. Yes we still have sales jobs here, but even those are being phased out thanks to the internet. Cutbacks will surely include a reduction in advertising (thank God!). What we tend to ‘produce’ now in the US are services and intellectual property (something large parts of the rest of the world do not even recognise). The MPAA and RIAA members really do not have much cost tied up in production so they can easily reduce prices along with the reduction in demand. Many service industries (like medical) are simply not discretionary spending - so they are safe.

What is really going on here and is it something to worry about? It will not be painless, but all I see is a reduction of this mindless consumerism that is destroying the planet. As technology progresses we will eventually have to face the prospect that most people will not have to have jobs - and I’m not talking about living on the dole either. There simply will be very little reason to ‘work’. It is going to happen and when it does we had better be socially ready for the consequences. Perhaps a bit of deflation will serve as a test of how we will deal with our inevitable future.

I was contacted several weeks ago about the use of some of my Moon photographs for a TV show. They were looking for photos of lava tubes. Of course, the most easily seen ones are collapsed tubes, but they did not care. They saw some of my photos while they were mining flickr. We exchanged emails and refined what they were looking for and I happened to mention that I also had some movies of the moon. So, I sent them several photos and an AVI. I’m not really sure what if anything they used, but given the fact that I just got an email announcing the premier of the show just moments ago, I can only surmise that something was use in production. The show, Sci-Trek: Mining the Moon, will premiere on the Discovery Channel Thursday, December 11 at 10 pm ET/PT!

A comparison between stacking in RegiStax (Left side) and AviStack (Right side).

I was recently introduced to a new and free (as in beer) stacking program and have been testing it. The program is called AviStack and it is written in a language called IDL. I’ve been programming in IDL since 1981 and have considered writing an image stacking/processing application with it for some time. Someone has beaten my to the punch and done a very good job in the process. One particular thing of interest is that there are free IDL virtual machines for most operating systems and so AviStack will run on just about anything. So the question is: Is it better than what is out there already?

We’ll see in a moment, but first, What is Stacking? Stacking is pretty much what it sounds like. Multiple images of the same thing are aligned and averaged together to improve the result. Averaging multiple images allows you to increase the signal-to-noise ratio of the image and overcome geometric distortions due to atmospheric turbulence (aka Seeing). See this post for a glimpse at the magic of stacking. The stacking process is comprised of a number of algorithms. The individual frames of the AVI have to be graded for quality and then aligned with a master image. When imaging a large object such as the Moon, the stacking process is done on several pieces of the image separately and then combined into the full frame. The reason for this is that atmospheric turbulence causes regions of the images to move relative to one another as well as warp and blur. Therefore, the stacking program has to look at every frame of the AVI and decide what, if any, parts of it are usable. Since the part of the image the program wants to use is probably warped/distorted geometrically, the alignment algorithm is another important task that has to be done well. Finally, the program has to keep track of what part goes where during the stacking/averaging process.

The image above is a side by side comparison of images produced by the “Industry Standard” - RegiStax (Left) and the new comer - AviStack (Right). Click the (or any) image to get the full resolution image to get the full impact of the comparison.

RegiStax has been around for a long time and is probably the single reason why amateur astronomy has made huge strides in the last several years. It is also free (as in beer) and is under active development. It works on just about any type of object and has a lot of controls to learn. The manual is borderline useful (being a translation to my native language) and so learning what each control does is more or less trial and error. Nevertheless, it is easy to learn. One of the most powerful aspects of RegiStax is its post-processing capabilities. It has a sophisticated sharpening tool based on Wavelets that is simply a must have for me. This comparison is based on RegiStax version 4.0.1.1.

In my comparison, the RegiStax result (Left side) was produced using the Human grading algorithm and 20 alignment points. The stack is the best 200 out of 1300 frames. The image was sharpened with Wavelets in RegiStax and a modest Adaptive Richardson-Lucy Deconvolution (RLD) in ImagesPlus. I applied a Gamma correction of 1.30 in PhotoShop.

The AviStack result (Right side) used a 32 pixel grading grid and had about 1700 alignment points. The TIFF output file was converted to RGB in PhotoShop so that RegiStax would read the file for subsequent wavelet sharpening. As with the other image, a mild 5-iteration RLD with a 3×3 Gaussian PSF was applied in ImagesPlus and the gamma adjusted in PhotoShop.

As you have probably already concluded for yourself, the AviStack result is clearly superior. The overall resolution increase is striking. Let’s dig into the details of the good, bad, and ugly and try to uncover what is behind the differences. To do so, I’d like to discuss the differences between the way each program approaches the problem. The steps I’ll consider are ‘Setup’, ‘Grading’, ‘Alignment’, ‘Stacking’, and ‘Post-Processing’.

Setup:

RegiStax Setup screen

RegiStax requires that you select features in the image you wish to use for grading and alignment by placing square regions of various sizes manually with the mouse on a single frame from the AVI. You need to be careful placing alignment points near the edge of the frame since in most cases the image drifts during the length of the AVI. Having an alignment point drift out of the frame is a bad thing in RegiStax. In my test case I placed 20 alignment points/squares (128×128) and then saved my settings. RegiStax does have an automatic mode to place alignment points, but I have never really gotten satisfactory results with that and always place my points manually. The maximum number of alignment points in RegiStax is 99.

AviStack Setup screen

AviStack first needs to determine the drift in the image over the course of the AVI. To do this the user selects two features in a frame and places small squares on them with the mouse. The program then runs through each frame of the AVI to calculate the direction and rate of image drift. AviStack also estimates the amount of distortion due to seeing. When completed the user then sets a few controls that will determine how the alignment points are placed. There are two controls that identify regions that are too dark (space is dark after all) or too bright (overexposed) where there is no need to try to recover information by doing an alignment.

AviStack Reference Points screen

Once the controls are set, a press of a button will generate many alignment points automatically. The program generated 1688 alignment points for my test case. Points can be added or removed manually if needed.

Pros and Cons: AviStack is certainly easier to setup primarily because the automatic alignment point selection works well. I don’t have to worry about image drift with AviStack either. Both have the ability to save the point placement information for easier reprocessing. I’ll give AviStack a slight advantage in the Setup category.

Grading:

RegiStax Grading screen

Having performed the Setup, hitting RegiStax’s Align button performs an initial alignment and quality estimation or Grading of the areas around each alignment point (within the specified box). RegiStax reads the AVI file multiple times, once for each alignment point. Using FFT-based algorithms to calculate rough integral pixel alignment and image quality, it chugs through all of the alignment points. This process is not fast, but it does allow the user to monitor the process and correct problems. The problems encountered most often are an alignment box drifting off of the edge of the frame and losing the ‘lock’ on a feature. The former is not a frequent problem if you place your alignment boxes carefully to begin with, but the latter is a problem I frequently encounter. This initial alignment and grading step is not a fast process.

AviStack Grading screen

The approach to grading taken by AviStack is very different. The user selects a grid size and the program divides each frame into blocks of that size. The AVI is read once and a quality grade is calculated for each box. During this step the user also selects how ‘Deep’ the final stack will be (i.e., how many frames to stack). This process is as fast as it is for RegiStax to grade one alignment point.

Pros and Cons: RegiStax is very flexible in how the image is graded. I have found that different grading methods work best depending upon the type of object (Moon vs a planet like Saturn) or the seeing conditions (poor vs good). AviStack is very fast at this step of the process, but you have to decide how deep your stack is up front and regrade the AVI if you decide to change your mind. This is not a problem since you do not have to go back to the very beginning (as with RegiStax). In RegiStax, the stack depth is set after grading so the user can look at the frames going into the stack before choosing the depth. The Grading step is a toss up between the two as a winner will be determined by whether the user values flexibility vs speed.

Alignment:

RegiStax Alignment screen

After the frames have been graded, the RegiStax user needs to decide on the depth of the final stack by setting a slider to the last frame (for the first alignment point) and hitting the Limit button. At that point two new tabs become active and I usually create a reference frame. This is an important step in the process as the reference frame is used for final alignment. The reference frame is created by quickly stacking some small number (like 50) of the best images (based on the first alignment point). This is a single point alignment. Stacking based on a single point alignment averages out the warping due to seeing, but does not produce the optimal image quality over the whole image. I almost always set my first alignment point to be a prominent feature in the middle of the frame. The user is allowed to slightly sharpen the reference frame with wavelets. Once again many options are available during the alignment step. Once set, alignment commences with a press of the Optimize button. Then you can go off and read a book, get something to drink, something else to kill some time while RegiStax reads the AVI several times (again) once for each alignment point. Alignment is done to sub-pixel precision by calculating the position of the maximum of the local correlation matrix. Again things can go wrong here as it is possible to lose lock on an alignment feature. RegiStax does have a few options to predict the track of features as the images drifts over the duration of the AVI, but it is not apparent that this prediction is remembered from alignment point to point. It could be used to help maintain lock.

AviStack Alignment screen

AviStack reads the AVI file once during the alignment step. For each frame, it uses the quality estimates for each quality grid box to determine which, if any, alignment points fall in a box that is good enough to be used in the stack. Those that do are aligned with respect to the feature found in a reference frame which is simply the average of all frames in the AVI. As the alignment is proceeding, the user sees the location of the original alignment points along with the location of the feature (both indicated by colored crosses on the image). Under all but the worst seeing conditions, the pattern of deviations should be reasonably smooth (correlated within a small area). Random fluctuations in the pattern of deviations indicates a problem which is correctable and reasonably well addressed in the manual. AviStack can track very low contrast features exceedingly well. As a result it can extract features in regions that you would otherwise think are featureless.

Pros and Cons: AviStack is a bit faster at this step in the process as well. It is also a very robust process. RegiStax is more likely to lose lock on a feature (although occurrences are not very common). In the lower left corner of the test comparison image at the top of this article you will notice that RegiStax had a problem locking onto the craters in the darker regions which resulted in faint displaced doubles. If you change your mind on the stack depth in RegiStax (or anywhere else in the work flow) you pretty much have to go back to the beginning. In some cases you should probably exit the program and restart it. The manual clearly states that canceling an operation can leave the program in an unpredictable state and that has certainly been my experience. The winner in the Alignment stage is clearly AviStack due to its speed and accuracy. Once again RegiStax does offer more flexibility, but I personally rarely use it.

Stacking:

RegiStax Stacking screen

RegiStax uses the location of the alignment points to divide the whole image area into polygonal regions with one alignment point inside each region. Using the best frames determined for each region it then interpolates the image (yes this means a potential loss of resolution for undersampled images) to the destination grid at each point in the region’s polygon and averages the frames together. Again in the usual RegiStax way there are many options to control and tailor this process to the user’s needs. One that is particularly useful is the degree to which adjacent regions are feathered to blend across polygon boundaries. The polygons can be significantly larger than the alignment box and so the quality of the region of the frames outside the box can be poor. At least with the real world seeing that I have where I live, there is significant smearing during stacking due to image warping within the large polygons.

AviStack Stacking screen

Here is where all of those alignment points ultimately pay off. Like RegiStax, AviStack stacks the interpolated sub-frame data into little polygon regions with a single alignment point in each one. The polygons are so small, however, that the blurring due to warpage from seeing is minimal. The result is that very fine detail is preserved and enhanced by uniformly accurate alignment.

Pros and Cons: Simply because of the sheer number of alignment points and the corresponding smallness of the polygonal sub-regions, AviStack will produce a more uniformly resolved result with a higher average resolution than you get from RegiStax. There is no aspect of RegiStax’s performance or flexibility that can outweigh this. AviStack wins hands down.

Post-Processing:

RegiStax Wavelet tool

You will need RegiStax simply because of its Post-Processing tools. The Wavelet sharpening process has to be seen to be believed. It is very powerful and a little goes a long way. It is very easy to over sharpen and produce an unrealistic image. This is very much a learn by doing process and it took me over a year to get it to do what I want. It was time very well spent.

AviStack has no Post Processing capabilities. The TIFF format you can save the image to is not compatible with RegiStax, at least for gray-scale, monochrome images. I usually convert it to RGB in PhotoShop before going to RegiStax for the Wavelets.

Pros and Cons: RegiStax wins by default and would put up an excellent fight had AviStack offered any tools.

Closing thoughts:
You will have to have both programs. AviStack is a fabulous new stacking program that produces astounding results and at breakneck speed. Although I have not covered it here, AviStack does not work for me with planetary images (at least on my system where I’m not using a V7.0 IDLVM, but rather IDLDE V6.3). The author assures me that it does indeed work, so I must be doing something wrong. I’ll update this once I get it working. RegiStax has a wavelet sharpening tool that you simply must use. RegiStax V5 is due out in the near future and I will certainly try that out as soon as I can and revisit this comparison. I have reprocessed and compared a lot of images with RegiStax and AviStack. A few of them from nights of excellent seeing and of detailed terrain produced close if not indistinguishable images. I have not seen a case where RegiStax produced a better image, at best it was just as good, but often AviStack’s result was superior (especially when it came to the uniformity of quality).

The current trend among the top tier lunar imagers is to stack only a few images. They will stack only the best 20 out of thousands to make sure they get the best of the best. I think AviStack is going to change that - a lot. Because of the very small regions it uses for alignment and stacking, users of AviStack can and will stack more frames. There is no penalty for doing so.

AviStack has a bright future. I would like to see the author publish an API and provide hooks where 3rd party IDL gurus like myself can add tools to the program. I’d like to add a mosaic tool, for example, and I’m sure someone would generate a host of post-processing tools like Wavelets or RLD or …

Both programs have features and short-comings that I have not covered that others will deem essential or show-stopping. I do encourage you to give both of these programs a try and let me know what you think.

I understand the argument that saving GM with a $25B “loan” might prevent an $80B shock to the economy, but who can say whether $25B will save them? GM does not produce a quality product that the consumer wants. If they did they would not be in trouble now would they? Then there is the other problem of the high cost of union labor. I’m sorry but I do not believe that a guy/gal with just a high school degree and a union job should be earning $100K+/yr putting parts of a car together.

If GM can come forth with plans/agreements that cut the cost of labor in half, build cars of the future (i.e., don’t rely solely on giant internal combustion engines), and do so with a quality product - I’d be in favor of a loan. Otherwise let them die and well pick up the pieces later. I still think they should be building wind turbines instead of cars.

2008 Sticker

I’ve cast my vote for president and was pleasantly surprised that I had virtually no wait to do so. I’ve been playing with some on-line electoral simulators recently and predict that the winner will end up with 338 electoral votes. The shocking news came this morning when I read and realized that Karl Rove chose exactly the same state-by-state outcome that I did. Very scary.

If you have not done so already - Go VOTE.

Flickr just changed the layout of every users home page. Some of the changes are good and some are good or bad depending upon what you like. One change that bothers me, personally, is what they have done with what used to be the “Everyone’s Photos” setion. It is now a more general Explore section with randomly rotating topics - one of which is the old “everyone’s photos” I’ve grown to love. I would like to have the option of what appears in the Explore section. While some may like the rotating feature, I personally just like to have “everyone’s photos” be there - always. Other users have expressed the decision to mix the comments we have made on the photos of others with the comment others have made on our photos. The old way kept them separate - which again, I liked.

Web 2.0 is all about the user & their experience. What this means is that we should get to choose what things look like. Flickr, don’t impose changes on us, just give us more choices. We like choice. Choice is good.