RTB aka Mass Surveillance on the loose

Real-Time Bidding (RTB)

In our society, nowadays, it would really be challenging to live without our smartphones, tablets or computers. If you are curious about privacy issues  or how your personal information may be handled when using those devices, you may want to read this article from the Electronic Frontier Foundation.

My iPad Safari is giving me this summary: 

"RTB is a process used to select targeted online ads, exposes personal information to thousands of companies daily. This data, called 'bidstream data', is easily linked to real people and can be exploited by data brokers for invasive purposes like tracking union organizers and compiling sensitive demographic information. To protect yourself, disable your mobile advertising ID, audit app permissions, and install Privacy Badger, a browser extension that blocks online trackers".

And if that read left you really concerned, you might be interested in what Janet Vertesi does to keep all that privacy-siphoning at bay. She offers suggestions galore for tightening your privacy when online and using your phone or even when shopping.  

Nobel Prize in Physics & Chemistry 2024 (Part I)

I will elaborate my thoughts in a later post. For now, this is meant as a stub addressing some misunderstanding of the relevance, or lack of, of "AI" (but not LLMs!) in these respective fields.

Let's start with the Nobel award for chemistry.

AlphaFold won the CASP competition in 2018 & 2020. The thing is that's a recurrent competition in Computational Biology seeking new methods/models for determining protein structure. These may have multiple applications, e.g, in protein design, where you want to optimize an amino acid sequence starting with a given structure. If I remember correctly, David Baker was the first in coming up with a new design and successfully synthesizing it -can't find the reference right now.

Biology not having a Nobel prize on its own makes it a candidate for either the Chemistry or Medicine categories. As I'm sure you know, Structural biology, as a part of biochemistry, is like "usual" chemistry but on steroids.

All this makes the award in chemistry meaningful to me: They all made important contributions to the field of Structural Biology.

It's hard to say the same of any impact of Hopfield, or even less Hinton, on Physics though. It's specially insidious in the case of the latter.

Let's now switch to the Nobel award for Physics that went to Hopfield and Hinton.

Hopfield, with his Associative Memory model, and McCulloc & Pitts, with their perceptron model, can be consider among the forefathers of Neural Networks (NN), in the shadow of Martin Minsky; Hinton made those NN "deep", and in doing so he ignited the "AI" fever of nowadays.

Hinton played first with a model similar to that of Hopfield, both of these based on what's called the Ising model in Physics. This was initially invented as a simple model of a ferromagnet. Let's see what how it works.

A ferromagnet is a material that behaves as a magnet on its own, after having it exposed to an external magnetic field. Kind of, "you charge it" and it becomes a magnet. It can "discharge" as well, again in the presence of an external magnetic field. Remember, the earth is a magnet, so expect some progressive discharge with time.

The transition from "normal" (non-magnetic) to magnetic state, or vice-versa, is the prototypical example of what in physics is called a second-order phase transition (PT). These are a special case of very "smooth" (in a precise mathematical sense) transitions. Many materials and systems, having nothing to do with magnetism, may exhibit such a PT. So the magnetism of the Ising model is, in principle, just a distracting, anecdotical feature. The ice-liquid transition of water is NOT an example of 2nd order phase transition, but a "less smooth" one, boringly called 1st order PT.

It turns out that in these 2nd-order phase transitions all the microscopic details of the different materials and systems become completely irrelevant for describing the overall features of the transition. There are differences on the exact value of when the PT happens, or how effectively it extends over the whole sample, etc. But, you can say that they all behave the same way if you "shift the graphs results" compensating for those numerical differences. To put an analogy, both alcohol and water evaporate at different temperatures, but both two transitions are first-order PT. You can perfectly overlap their graph results if you rescale them appropriately based on those numerical differences. Think of all instances of the character "s" in the previous sentence. They all are located at different positions along the sentence. However, you'd say "it's the same "s"' because you can "rescale" (translate) their position so as to overlap them perfectly. The rescaling you may do for second-order PT is different than that for first-order PT. This is a type of symmetry difference that puts those two types of transitions apart.

OK, so we have a ferromagnet that gets (de)magnetized in the presence of a magnetic field. Can we model this transition so as to convey those "universal features" of it? The Ising model is the first and simplest of models doing just that. This models the ferromagnet as a collection of "spins" that can be in only two positions, up or dow. Think of them as microscopic magnets having their "north" either up or down. When all spins are randomly up and down, the material shows no magnetization. But if all were pointing in the same direction, the piece of macroscopic sample you are holding in your hands would behave as a magnet.

Incidentally, Physics has a systematic procedure for removing all microscopic details of a fine-grained model. In doing so we obtain what's called an effective Hamiltonian (aka, model), e.g., capturing only some basic, coarse-grained features of the system. Clearly, any such model, by its simpler nature, may describe other systems of a completely different nature say like a physical system, a geometric problem or a combinatorial one. The Ising model can be described as such an effective "model".

As we often do in Physics, we describe the state of a system as that of minimum energy. That is, finding the properties of the system entails solving an optimization problem using a given model. For high temperatures all spins point randomly and the minimum energy of the Isign model gives exactly that, i.e., no magnetization. But fir low enough temperatures the minimum-energy solution of the Ising model shows practically all spins pointing along the same direction, i.e., spontaneous magnetization.

There you go, we used an optimization technique (mathematics) on a model (mathematics too, but with inspiration in Physics) with the goal of describing a phase transition (physical phenomenon).

Say you improve the optimization technique, f.i. by making it 10x faster, or tweak the model so as to describe a different material, but keep the goal of describing the PT. Clearly, you would still be making a contribution to physics.

Use the same technique and model but aim at storing and retrieving an image...and you are NOT making a direct contribution to the understanding of physical phenomena. You would be contributing to... you guessed it, Machine Learnig.

Emphasis in "direct" in that last paragraph. If you want to argue that all our knoledge is connected or that our reality might be digital, that's a whole different discussion that you are a free to pursue.

Let's recapitulate. The fact that you can use methods and models initially developed for Physics to study problems in domains far away from it may be surprising and lead you to think there is some substance to the award. But, as you can infer now from the previous discussion on the Ising model, on one hand, that's a feature of Physics and its language, Mathematics. On the other hand, that fact leads to a, currently still a philosophical, question of what is the ultimate object of study of Physics.  Landauer's principle directly connecting Physics and information (erasure) opens the door to interesting speculations, as it makes information a "traditional" physical observable that can be measured in principle.

So, no, I don't think Hopfield, and specially not Hinton, deserve the Nobel prize in Physics.

Note added in proof: If the Nobel committee just wanted to "sell" us something I can understand. That's what I did in assigning labels to this post.

https://www.nobelprize.org/prizes/physics/2024/advanced-information/

Insomnia: van Aubel's Theorem

I woke up in the middle of the night and stumbled upon this post in mathstodon about a simple theorem in Euclidean geometry. So I thought it was a good time to dust off my Geogebra app.

Van Aubel's theorem states that if you start with a convex quadrilateral and construct a square on each side, externally to the quadrilateral, then the line segments connecting the centers of opposite squares will be equal in length and perpendicular to each other.

The following images provide a synthetic proof of this theorem. The line segments mentioned are colored in green and red.

The green and red circumference cross on point $O_1$, which is the center of the blue circumference. By construction, the blue circumference contains points $M_1$ and $K_1$. Hence the red and green segments are of equal length.

The purple line is the perpendiculat to the green line that passes through $P_1$. As can be seen, the purple line contains the red segment. Hence the green and red segments are perpendicular.

This concludes the proof of this theorem.

I much clearer proof is following diagram.The red circle around $M$ contains the centers $I$ and $K$ on its diameter as well as point $N$, where both (green) segments joining opposite centers cross, thus forming an inscribed triangle, which proves the segments being perpendicular. 

Finally, each segment can pivot around $I$ and $K$, respectively, meeting at a common point $O$, around which they can pivot again drawing the large red circle that contains $I$ and $K$. This proves both segments are the same length.

The last diagram shows that, in essence, Aubel's theorem hinges on a similar property for triangles. Here $H$ is the midpoint of a side. The yellow circle, centered on the midpoint between $F$ and $G$, shows that the two green segments are perpendicular, while the purple circle, centered on $M$,  proves they are the same length.

Concurrent Programming: Linux Memory

While I finish the blogger version, you can have a look at the original entry in my GitHub account.

TL;DR:

One practical take home message is this:
When using top in Linux, it's not meaningfull to add the RES value of different processes as that does not amount to the actual, combined physical memory used by all of them. It's also not meaningful to add the values of USED for different processes; neither is it to add the values of RSsh: They do not account for independent chunks of physical memory; we are overcounting in general!
See the examples in that post.

What Is Science?

 We all "know" what Science is...at least at we have a pedestrian working definition of sorts: based on experimentation, models and theories need to explain known facts but also predict new ones that then must be confirmed...by experiments. Some say this as they must be refutable -as in the opposite of not even wrong.

Here I want to suggest another, hopefully equivalent, idea. Something that has bothered me for some time now is that naive realism that I feel lurks under expressions like the following attributed to Max Planck

An experiment is a question which science poses to Nature and a  measurement is the recording of Nature's answer.

The following ideas are closer to a different quote attributed to Carl Sagan

Science is a way of thinking much more than a body of knowledge

So, What is Science then?

Looking through the window you might say "the trees are tall, the grass is green, the hill is smooth, the dairies are white and yellow,...".

That is poetry, literature. It's another approach to reality, to learn about ourselves in relation to others and to Nature. And it is a necessary one.

But it is not Science. Nor those are the only type of sentences we can come up with.

Here some different statements: 

• "When the sun shines flowers bloom, grass grows and becomes greener, trees get more & greener leaves,..."
• "When humans constantly tread on the grass, the grass dies and the area becomes sandy & arid"

Wait, "dies"?? What's that? What is death? A possible answer would convey the following idea:

• "When we provide food to a living being it grows and multiplies. Eventually it stops doing this and we say then that it died."

All of Science is in essence a collection of sentences of the type

When we do/it does this, it happens that

In order to understand the it, we (let) apply "transformations" on it -the object of the transformation-, and "collect" the output -which itself is an it! 

I saw a parallel with Category Theory: Objects are secondary; what matters are the morphisms. These tell us everything about the objects.

In Science, experiments are but transformations.

Stated like that, it reminds me of the warning of Carnap about Science: it's a combination of experiment plus theory. Theory as a mental construct/model.

Q1: What is theory in our view?

Ans: Our view doesn't change this, but it may give it a spin. Theory & models are but a mathematical guide for us to do

• Actual experiments
• Gedanken experiments

And thus anticipate the outcome (or rather, how the outcome behaves). 

As these mathematical constructs can themselves eventually be expressed within CT, we could justify the claim 

Science is about applying & combining transformations

The inputs & outputs are themselves understood as transformations, thus it seems redundant to include them in the above claim.

Q2: This still seems to miss Carnap's observation that Science is not a collection of facts. A better formulation?

Ans: Science is about applying & combining transformations...and the talk about it.

We mentioned transformations as synonyms for actual experiments.

The talk about it is our on-paper, mathematical experimenting plus the verbal talk and descriptions. The verbal talk can eventually be fitted within CT.

Q3: Usually one stresses the fact that scientific method must be 1) predictive and 2) refutable. Are we missing this?

Maybe we can introduce a "why":

We experiment and talk about it in order to be able to anticipate (predict).

Consider the following:

"If you drink this potion and repeat 10 times these words, the gods will vest you with the power of invulnerability against bullets"

It talks about transformations (drink potion, utter words), has a prediction (invulnerability against bullets) and is refutable.

Our goal to anticipate makes us try ti make sure it works -instead of just take for granted-, and thuscto refute it. And it turns out to be false.

Maybe then the "why" and its answer "to anticipate" comprise those common two traits.

We come then to the following statement:

Science is about applying & combining transformations and our talk about it in order for us to anticipate (the outcome)

What I find so much appealing about Category Theory is that it applies the same approach to study mathematics itself. Instead of mathematical object&structures it shifts the focus to the morphisms (transformations) on & between them plus the properties of combinations between those morphisms.

For example, and this is a general modus operandi, not something CT, if we want to understand different types of graphs we could run a set of stochastic processes on them. One same stochastic process may proceed differently on different graphs. Ultimately this allows us to distinguish and classify those graphs.

That "talk about it" is thus a talk  in terms of applying and combining mathematical transformations. 

Ingredients for a Successful DevOps Practice

...At least three basic ingredients.

If I would have to pick what I think could be three basic ingredients for becoming a successful and valued DevOps, it would be the following.

First and foremost one has to have a passion for automating things. It's the "hate to do things twice" attitude. But this is not software development per se. The goal here is to abstract away a process in a manner that gives us

1. the possibility to control, test and run it with a range of options and
2. to grant us a meaningful birds-eye view of that same process that will facilitate the discussion of it and of our infrastructure as well as eventually lead to further improvements of it.
   

It is automation as a way to gain a better overview of the infrastructure, its uses, and the needs for the business

Second ingredient, the "how you achieve automation of a process". Make a real story of your Jira story! Don't shy away nor be lazy to add comments full of details. This is the place where you don't need to worry much about formatting, yet you have all the tools to insert screenshots, links to other boards or documents, link it to further stories, tasks or even epics, explicitly raise the attention to other people that may or not be in your team, add snippets or even full scripts code, etc. Most importantly, it's a single stop shop where you can

1. First just jot down an idea of a project
2. Draft the requirements and goals
3. Annotate some initial tests and investigations for solutions
4. Divide the story into tasks and littel milestones
5. Go to step 2 and keep improving until a satisfactory solution is found
   

Again, you get to do all that without worrying about the eye-candy of formatting not even constantly writing full sentences.

Clearly, the story is a living object that may evolve in content as you advance in its solution. And the good news is that it is always there.  In the short term this helps gain accountability as well as keep in check our logic. In the mid and long term this attitude serves the traceability of the decisions and solutions taken. Others will certainly benefit greatly from it!

The third and last basic ingredient I would highlight: Show due reverence for the logs. Any scientific field, be it Physics, Math, Engineering or even Language or Humanities can be understood as a composition of processes. A network of maps each with its inputs and outputs: the output of each process is the input of the next.

The logs of your automation include any of its required outputs (say updating a bucket, building a stack, triggering a pipeline, etc.). But it's vital that its output be seen as a story in itself: Reading the logs of your scripts should immediately provide us with a clear view of a

1. Full path of your script as it was called
2. All values of all variables used
3. timestamp of each step and two cents of what it's doing
4. which function in the script is doing what at that time
5. which function ended in error and in which line of the script

And that should be done with a formatting that is

• consistent
• easy to parse

I can hear some of you already shouting "how does this apply to Terraforming or CD/CI?". 

Well, you better structure your Terraform code into modules -yes, beyond the basic main.tf, variables.tf and terraform.tfvars. You see, It isn't then any different! And the whole CD/CI...There is a reason why the mathematicl concept of a map (for CS minded readers, think of lambda calculus, or a language like Lisp or Ocaml) is so powerful: It conveys the most basic idea of a path/process, among a plethora of other technical details. And that's what our pipelines are.

Of course is easy to keep adding ingredients: Eagerness to constantly learn and use new technology, embracing the KISS rule, be quick in documenting things, ... But if I'd have to pick a fourth one I would first go with a 

Trust the adage "good is better than perfect".
 

Although I first heard it as "Perfect is the enemy of good". You get the idea though.

Tutoring

 Tutoring

I asked about an update on how a former student finished their course last June.

"X's math course did not go well: he ended with a 70. Way below his ability. Not sure what happened but we are hoping that the return to in person learning will help him regain his footing academically. "

It's not satisfying. 

I'm definitely sorry to hear X couldn't keep that higher mark that he was getting during the course.

The course structure didn't help. Yet, my task was to at least keep him at that higher mark range. So, that's on me -at least in part and to whatever extent I can influence that.

 

From the point of view of the student, every tutoring puts a little burden on them to try to match the explanations from the teacher and the tutor relative to the same concept. 

From the point of view of the tutor, the task is in part to find the adequate strategy to deal with the usual first questions the student will raise: "I don't understand it", "I don't know how to do it" or "I don't know what I'm supposed to do". 

 

More often than not, the student knows it. The problem instead lies on making the connection between a problem's statement and their knowledge of the subject. The reason may range between two ends. 

 

On one end the student knows the mechanical steps, "the how", but doesn't quite understand, and even feels at lost with, why those steps are as they are or what they mean. This makes it hard to solve open-ended problems, where inquiry and thinking are the major skills leading to a solution. Often the latter is simple in terms of calculations required. The hard thing is to come up with the idea of calculating it that way.

 

On the other end, the student has some (a reasonable?) understanding of the general idea, but lacks in enough concrete  practice  (i.e., hasn't practiced enough, hasn't done enough exercises yet). Two examples may help: Adding fractions will always show up in math, after they see it in grade 6/7. A weak background on it will become a frustrating drugging force later on when facing Math in grades 10 and beyond. A different case that always shows up is at the beginning of a new topic. They may have worked out some examples in class, but the concepts, while making sense when explained, are still not completely clear in their minds, and even less so are the calculations involved.

 

I see Y as falling closer to the first case, while X, however, would lie not far from there, but somewhat more towards the middle between both ends.

 

I'm thinking that in cases like X special care should be taken. Finding the right balance between working on the concepts and working out cases is tricky.

Now I think for X it would be best to keep much more concrete on the task at hand, the exercise to solve. Furthermore, we should deviate as little as possible from the procedure followed in class by their teachers.

Despite all effort one may make, sometimes the differences between the teacher/tutor's approach and the needs of the student can't be brought enough in sync.  Changing the teacher/tutor may then do the trick.

 

It may well be that oils boils down to me emphasizing more problem solving that concept discussion.

Following the motto of "Be pragmatic!"...as that's the simplest, and most common measure of what one has worked on with the students. 

 

Feeling a bit frustrated. Let's see what the parents reply and if I can get more feedback on what happened...