On a less serious note

So, I realised just a few minutes ago that I haven't written about chaotic attractors in all their splendour. An incredible oversight on my part. These objects are achingly beautiful and intricately complex in form. They arise in phase space (I believe I have covered this) in certain parameter regions of systems. Their fractal geometry means that dynamics is incredibly sensitive on them and while trajectories will follow the path of the attractor (duh! attracting :-) ), there is no predicting which exact part of the attractor it will land on and follow round or ultimately where it will end up. The attractor is fixed, the trajectories are not. A nice succinct explanation is available here.

The one that ties in closest to my work is the Lorenz attractor. The three dimensional system for describing turbulence gives birth to an amazing structure. This attractor is where the 'butterfly effect' expression originated. The attractor itself is geometrically between two and three dimensions and winds itself around two points.

The image (from wiki) shows exactly this. Where the trajectories 'cross' in this two-dimensional visualisation is actually where they layer over each other - manifolds cannot intersect (the unstable manifold exactly describes the attractor). The view is better in 3d. The fractal (Hausdorff) dimension of the attractor is about 2.06.

Other systems with strange attractors are the Rossler system and Henon map. This page gives a nice description and a few images.

As a side note: for those of you who read and enjoyed 'Harry Potter and the Methods of Rationality' or 'Luminosity' I would recommend this strangely named (but you'll understand all too soon) 'Baby Eating Aliens' also from the Less Wrong family and the (there is a theme here, no?) 'Harry Potter and the Wastelands of Time' if you like a bit of complex, epic writing with lots of action and magic!

Enjoy! Also, something to look forward to: I hope to have my poster available for you lot next time I post. :D

Doh! Almost forgot. One of the lecturers at my university (Hinke Osinga) actually crocheted the Lorenz stable manifold! If you want to know how, read this!

Incidentally, my project supervisor is the Bernd Krauskopf mentioned in the article. And yes, it is one of their Christmas decorations. ;)

Horseshoe Chaos

While I struggle with supposedly impossible inversions, I'd like to introduce you all to Smale's horseshoe. This object is where the definition of chaos originates yet it is a relatively simple idea.

You start with a square. Squish is down and pull it out to the sides then bend it at the halfway point so it looks like a horseshoe. Then replace it over the square so that the two lengths form vertical strips on the square. This is the basic transformation of the horseshoe map. It is also invertible. Take the bent strips, rotate back to horizontal, and unbend it then squish it in from the sides and pull from the top and bottom then you get the square again. If you do this inversion again you get strips intersection horizontally.

Repeatedly performing the forward transformations you get more and increasingly thin vertical strips: 2 strips become 4, 4 strips become 8 i.e. each strip gets two thinner strips within it at each iteration. Doing the same backwards, you get lots of increasingly thinner horizontal strips. The points in the strips are the points that remain in the set at that iteration. Most points leave after even just a few iterations in either direction.

If you overlay the two directions, the points at the intersections that remain in the square under all iterations (if you iterate infinitely in either direction). This is the invariant set. This set is a Cantor set (disconnected and unstable fractal).

If you labelled each point with 0 or 1 at each iteration you could describe the positions of all the points in the square e.g. .0101 means a point that is in the 6th vertical strip after four iterations (the right hand strip of the left hand strip of the right hand strip of the left hand strip in the first iteration) or

If you can see that...sorry if it's too small.

Since you can do this in both directions, the whole dynamics of any point in the square can be described by it's bi-infinite sequence of positions.
Knowing this we can construct any orbit we like and we know it will describe at least one point in the square.
So we can make a periodic point: ...010101.0101010... which alternates from the left to the right. Since we have infinite lengths in both direction, we can make (uncountably) infinitely many of these periodic points.
Or we can make a non-periodic point: ...010001101100000.1010100110111... by, for example, 'counting' in binary as above or just by adding the wrong value to a periodic point i.e. ...01010101.0101011...is non-periodic. There are (uncountably) infinite of these too.
Among this set of non-periodic orbits we can find at least one that comes arbitrarily close to the invariant set. This is trivial since you can take just part of the sequence that describes a point in the invariant set and it would come as close as you like!

I hope you are following since I have just demonstrated that Smale's horseshoe perfectly defines chaos.
Chaotic motion must consist of the following:
- infinitely many periodic orbits
- non-periodic orbits
- dense orbits - ones that come arbitrarily close to any point in the set.
There tends also to be the condition of sensitive dependence on initial conditions but this can be shown through the dense orbits - you can take two sequences that are the same for any arbitrary iterations and in the next they can end up in two different strips i.e. they may start very close but can end up very far apart.

Voila!

Strogatz does give a very basic introduction but for a more detailed approach read the first chapter of "Elements of Applied Bifurcation Theory" by Yuri Kuznetsov. Wiki also has its own explanation.

How many drafts...?!

So here we are, nearing the end of my project. Well, actually there are six weeks left til I hand in my thesis but it doesn't feel like it. It feels more like two. :S

I've been asked to do two drafts already. A third one is coming up next week and I'll redo the current one into a fourth. After that they combine into a fifth. Then will follow a sixth from recommended improvements. I'll probably create another couple just for kicks! So that's at leasts one a week...a draft roughly for each month I've worked on it. I'm drowning in them!!

Ah well, at least it means I have shiny pictures for you this week. My draft task this week was to tell the story of one side of my research (decreasing lambda) as purely in images as possible. Obviously the details will be missing but I hope you enjoy the obvious progression and changes in images as lambda varies.
My image report draft is available here.

In other parts of my Masters world I am winding down on lectures and winding up for revision ready for exams in nine weeks. I want to try something new for my revision this year - sort of step it up a level. If anyone has any recommendations that would be very welcome. I have a few ideas but would like to augment these with other influences.

 I also have to produce a poster and a website for my Masters project but I'll share them here first to get opinions! I will of course add a link to this blog in my website (cheeky I know!); maybe they'll give me extra marks for initiative and knowledge sharing never mind the pretty pictures!

Weird Manifolds

So this week, I've been using the new software (which is working well now) to calculate the invariant manifolds of the map I've been researching. The best way to think of manifolds are the shapes/curves that the system points hop around on as they iterate under the mapping. There has to be saddle points in the system in order that the stable and unstable manifolds can cross and then be calculated. The calculation to find the manifolds requires tracking backwards in time the path of the saddle point...which shouldn't be possible in a noninvertible system. This is where the weird stuff comes in especially in the case for my map.

For some points, there are no points to track back to or four and for all others there are two possible previous values. Currently the software can calculate two possible values and then for each carries on the path back taking either the 'positive' or 'negative' - whichever is the most likely to be the previous value. It does factor in the no pre-images case but not yet the four possible values. That is part of the work I still have to do this year.

It helps to see some images so here are some I calculated just this week!!

The red line is the unstable manifold and the blue is the stable. The point where they cross is a saddle point in the system and the other cross is the repelling point. I've only shown the positive manifolds (one side of the possible previous values) but the negative manifolds are just the reflection in the x-axis.

A Pause

There are three very good reasons I haven't really got anything good to talk about this week:

1. I have been wrestling with 32 bit compile on 64 bit architecture in order to get the essential program I need for the next stage of my project installed, compiled and working. It has only just been sorted with some fantastic work-arounds.

2. I have had every afternoon in the last two weeks filled with back to back lectures, mornings with exams and a presentation (which I mentioned last time) as well as work for those said lectures.

3. I have managed to slice open (a bit) the knuckle on my right hand thumb on Wednesday and thus typing, writing and pretty much anything is pretty hard now that I have to do it all left-handed.

Poor excuses, however next week I will dazzle you with fantastic images relating to invariant manifolds. You can read about them if you would like something to do that is more productive than reading this.

Pwetty Pictures and a Presentation

So, this week when I thought about what to blog I left it pretty last minute. This means that instead of an (usually very interesting) insight into chaos I will share with you the presentation I gave on Friday for my project to students and lecturers. I have left the notes attached so you can follow what I spoke about as well.

One moment from someone else's presentation I would like to share is this:

Brilliant!!

The presentation itself was about business intelligence which does not really interest me but the image brought some light humour to the talk. 

Nonlinearity gets all the fun

Nonlinearity is inherent in most systems before you make an assumption and reduce the beautiful complexity down to a boring bog standard linear system. There is a section in Strogatz that shows probably the most interesting thing you can do with linear systems (Chapter 5 section 3 page 138 onwards). And I only read it once because linear systems were dull and pretty repetitive long before A level.

Nonlinearity is pretty easy to understand. Take for example square numbers. You can make 4 by either multiplying 2 and 2 or -2 and -2. This means that, under the square operator, 4 has two (well, four really, but two are identical to the other two) possible previous values. This makes squaring a number nonlinear because tracking back you do not know whether two 2s or two -2s made the 4 you see at the moment. This particular brand of nonlinearity is called noninvertibility. You still following me?

Good. Right, to take this further we call the previous values of each number pre-images and the operation, a mapping. Now the quadratic (squaring) mapping that I am examining is:

what happens to z when a=2 and I vary lambda. This mapping does funny things depending on what I set lambda to. If it is between 0 and 1 then it behaves just like the squaring operator - each value of z has two pre-images as I iterate the map but in a circle radius=1-lambda around C the values have no pre-images. Now if I make lambda greater than 1, some values have two pre-images as before but the z values in the circle now have four pre-images.

Still with me? We can take this further but placing this property on curves not just points. A curve is just a line on a graph (an infinite set of points). Instead of looking backwards as we did with the points, we instead go forwards and see how the curve changes. It's first image is a circle, say. This curve then becomes the pre-image of the set of points of the circle under the mapping. And so on, forwards in time. It helps to look at the image below:

You can see the curves collecting on (heading towards) one curve called a manifold (it's attracting). Now consider the relationship of the points and the curves. Because a curve is just a set of points. With noninvertibility inherent in the system, what happens it you take the manifold and reverse the mappings for all the points? Wild Chaos! Tracking the possible points and curves and how and why they interact is the next stage of my project - at least for certain ranges of values for C, a and lambda. This limitation is purely because my project has a time limit of a year.

I am currently working with a PhD student, Stefanie Hittmeyer (who incidentally produced the image above) who is working on the full problem in all its scope. She has been focused on the manifold part of it for the moment while I have been examining the fractal structure side. My work nicely compliments hers. This work is on the fore-front of chaos research. Now it may seem that all this has little value in the real world but the great thing about nonlinear and chaotic systems is that it can accurately mimic real world systems. This is a major advantage over linear systems because when taking a real world problem you have to reduce the picture you are looking at so you can describe it using the simplest tools. With this kind of research, no such reduction is necessary and you can understand the system in it's full complexity. So maybe it'll turn out that neurological signals in the brain will turn out to have similar properties to the mapping that I am examining and my research will contribute to a fuller understanding of how we came to have consciousness and help neurosurgeons get better at fixing the biochemical circuitry that makes us who we are. Or maybe, and more likely since this mapping is a reduction of the five dimensional Lorenz model, it will just help meteorologists get really, really good at predicting weather patterns.

Who knows? It's sort of like the laser - a solution created with no problem that has now become integral to our lifestyle. Just check out how many things it is used in! I hope you managed to follow all that to the end. If you have any specific questions please feel free to email me but I would much rather you follow that urge to take a deeper look yourself (even if that is just reading all the wiki pages). It's way more fun!

Oh, a promise

I also promised a friend I would post a picture up this week so here it is:

This is from the edge of the Mandelbrot set - originally from Wikipedia
And an example from my own research will be uploaded next week.

Rationalising Chaos

Since I have been on my Christmas Break, I have only just started getting back into the thick of my research. Thus, this week's post will be on a related interest of mine. Rationality and luminosity.

I have been attempting to find order from chaos within. I have always known myself quite well but there are times when, and I'm sure you will have had a similar moment, I have thought "What was I thinking?!" having just done or remembered something incredibly stupid or damaging. I have often wondered when I would get around to finding out. I started my path to luminosity through rationality when I began reading "Harry Potter and the Methods of Rationality" by Less Wrong who took the pen name from the website which is "a collaborative blog devoted to improving the art of human rationality". From that website I discovered Alicorn who had written a series of blog posts about luminosity and a subsequent rational fanfiction of Twilight called "Luminosity" .

These encouraged me to really being to explore who I am and how I do things. To examine my motives and actions and be able to explain to anyone why I do the things I do and think the things I think. Living this way prevents one from lying to oneself in order to imagine they or the world around them is they way they want instead of the way it is. If you too want to explore this, I would suggest reading the stories first to understand what I means to live like this - to stand out because you refuse to gloss over the parts of reality you don't want.  The world is chaotic but not really as irrational as it seems.

By saying "gloss over reality" here I don't necessarily mean in a politically active sense in fact I tend to ignore politics because every side glosses over their reality and never say what they really mean - even taking an opposition to politics means you a glossing over the reality that democracy has to please the masses; I try to take each case individually on merit instead of generalising. It could be as simple as packing a first aid kit in the car, not because you know you will need it on that trip but because you know that things happen and it is better to be prepared for the worst instead of saying to yourself "by packing the first aid kit I'm tempting fate and someone will get hurt so I shouldn't pack it" just like thinking that if you take an umbrella with you, it is more likely to rain, or by not going to the doctor, you're not really sick and the problem will go away by itself - twisted and irrational logic.

By accepting the reality that everyone gets sick at some time or another, that it will rain unpredictably in England and that people can get hurt doing the safest things, you can prepare for these realities and maybe avoid the worst.

Quantum Time Series

Currently I'm reading a book on quantum physics ("In Search of Schrodinger's Cat" by John Gribbon) and I was struck by the similar history of development it has with chaos theory i.e. no-one wants to admit it's really true and almost no-one really understands it. It has taken almost 100 years for quantum theory to be discovered, accepted and still it is only fully taught at university level after student have already been ingrained into thinking that electrons whoosh around the nucleus. I remember each level of school (middle school, pre-GCSE, GCSE) I took chemistry, I was told that everything I had been taught previously was wrong and too simple and here is the real thing! It got to the point that when I was choosing my A levels I did not pick Chemistry. I figured they would just lie to me again and I was better off learning the truth out of university textbooks. So I took Philosophy instead. I wonder whether others felt the same. I just wished they could have taught it correctly from the beginning and get over the fact that most 10-16 year-olds might not get it. Some will! Maybe in another hundred years quantum will be understood by enough people *cough* politicians *cough* that it can be taught straight from the beginning and not require almost a reprogramming of minds. Perhaps the same progress will happen for chaos theory. One can only hope.

This week my paper is on interaction of populations of competing species. Available here. It's quite a nice paper (longer than the last one). The purpose was to analyse how three equally competing species interact to change each other and their own population size over time. The time series plots provide examples of how systems require a certain amount of time to settle down to the final state. The equilibria that I talk about in the paper are fixed points for the system. They are the stable behaviour of the species interactions. So the species may start at different values (with the same parameters) but, over time, they will settle down to a fixed point (or periodic behaviour, or chaotic behaviour).

Taking the example of the pendulum again, when you set the pendulum in motion it will oscillate a few times and then settle back down to the lower central point. This is because it is damped by drag. This behaviour can be seen mathematically as a perturbation of initial conditions away from the stable equilibrium but over time it will return to that stable equilibrium, that stable position where it can hang without oscillating quite happily until the end of time. This stable equilibrium attracts all motion of the pendulum towards it so it is called an attracting fixed point.

Now opposite that stable equilibrium there is an unstable equilibrium. Right at the top of the circle that the pendulum could swing through. At this point the complete opposite happens to the point at the bottom. It repels all motion from it. A pendulum cannot hold itself up until the end of time (not with gravity anyway). So at the top we have a repelling fixed point and at the bottom we have an attracting fixed point. Mathematically the system makes sense.

Right, so now the easy bit is done, I'll add something a little harder to the end. You might be about to comment, "So what is happening when the pendulum in a clock oscillates on and on". So I'll tell you...

That is when the system has a period two oscillation if you set drag  to zero (it is not exactly possible in the real world because drag does affect it due to not quite being zero which is why you had to keep winding them up). This means that when you set the motion going, you can get the pendulum to swing for the rest of time side to side. The distance and time period of the swing will be exactly the same for every oscillation. This is a stable period two oscillation and has a pretty sine or cosine curve on the time series plot.

Pendulum's can display chaotic motion. Just check this out. And each time you set it in motion it will swing and flip in a completely different way thanks to the effect of chaos - sensitive dependence on initial conditions!

A Biological Twist

Since I am an engineer you might wonder what I mean by the title. As I said previously, chaos theory is applicable in every science. Biological systems especially exhibit chaotic behaviour. The system under examination today involves DNA within a cell. Some background on DNA and the DNA - transcription - mRNA - translation - protein cycle is available in an interactive form here.


My document for this week is available here.

The Goodwin Oscillator describes a gene's behaviour when it is self-repressing i.e. as it is transcribed and translated it recognises if it is already present at a certain concentration and prevents any more being made. Exactly how this happens biologically I have very little idea. As an chaos mathematician and an engineer all I need is the equations used to mathematically describe the system. From there, as you will see in the paper, I can build a view of what the system does under different parameter values.

In the paper I have used two software packages: MATLAB and XPP. XPP is the best for newcomers to the chaos scene if you want to play with systems. A tutorial and the free download are available here. MATLAB is a (definitely not free), matrix based, mathematical tool provided by Mathworks. A lot of people don't like to use it but I have worked with is so long I can now avoid most of the annoyances and can get it to do what I want now. It is good at crunching lots of numbers for chaotic systems and displaying their behaviour diagrammatically.

In the paper I have focused on the bifurcations of the system. A bifurcation is a point where the system qualitatively changes behaviour dramatically and instantaneously. A steady system can suddenly become oscillatory under the right parameters. In the paper the system of gene self-repression in the cell is bistable - it has a period two oscillation; it oscillates between two values -  for all parameter values that I could calculate. The unusual behaviour comes just before that - the behaviour before it settles down to bistable for small x/alpha values is rather strange even for chaotic system. If I could have furthered the paper I would have tested that area more thoroughly and worked out what happens there. However, I was limited by time and pages.

The best thing that the system I examined in the paper shows is that even very simple systems can exhibit highly unusual behaviour as well as perfectly understandable behaviour. It also demonstrates how understanding the mathematics leads to better understanding of the biology. Even without knowing the biology or chemistry to flesh out the details I can tell you that a self-repressing gene will allow itself to be 'switched on' when the concentrations of the mRNA and protein reach certain low levels and will then 'switch off' itself when the concentrations reach certain high levels with the levels being determined by the particular physically characteristics of the system (parameter values). I can tell you that interesting research (in both the biological and the mathematical sense) would be in the area for very small parameter values. I can tell you that I feel that I now understand DNA transcription and translation behaviour better because of this research.

You might think that what I described happening is perfectly reasonable and of course is what happens! But the paper can specify which cells (if exact enough measurements can be taken) will display which behaviour for that gene. And it can specify which cells will not. And it can go further and build from the simple, constrained system to a more complex one taking the mathematical and biological understanding further. Biology is one field that is now accepting chaos theory with open arms because it opens new doors to research and furthers understanding of biological phenomenon by being able to generalise as well as specify. While the field may have arisen from Physics, it is Biology that is currently receiving most of the benefits. In my department, four out of fourteen lecturers have research specialisms in applying chaos to biological systems. That's 28.6% of the department - almost a third.

My final thought for you is for you to consider, if even genes behave chaotically, what other biological processes could do so also?

Further reading for bifurcations: Strogatz (if you have it) covers them really well in ch 3 (basics) and ch 8 (much more interesting!) also a good starting point is the NLDC web page under 'Nonlinear dynamics in action'.