[{"content":"Selected posts, projects, and short notes on statistics, mathematics, and practical Python work.\n","date":null,"permalink":"https://blog.leokrglv.net/","section":"Home","summary":"\u003cp\u003eSelected posts, projects, and short notes on statistics, mathematics, and practical Python work.\u003c/p\u003e","title":"Home"},{"content":"","date":null,"permalink":"https://blog.leokrglv.net/posts/","section":"Posts","summary":"","title":"Posts"},{"content":"This is a small note on how to use a GPU-based sampler on linux with an AMD GPU.\nWhen using PyMC, it is possible to use external samplers, that increase the sampling speed. For CPU-based samplers, the best-known choice is the Rust-based nutpie.\nIf the underlying model has a high number of levels and/or any type of significant hierarchies, one may want to consider a GPU-based sampler. For that, the common solutions are jax-based numpyro and blackjax samplers. To properly perform them, we need a dedicated GPU. As of today, the most solid ecosystem, even on Linux, is, of course, the Nvidia\u0026rsquo;s CUDA. So the most frictionless way to make the common libraries is using the CUDA infrastructure. I am, currently, an AMD user, so making common libraries work to use GPU\u0026rsquo;s capability, is often not trivial.\nMy goal here is to use my newly installed AMD GPU to use efficient JAX-based samplers.\nAfter considerable effort of googling, reading AMD\u0026rsquo;s guides, github issues and using LLM\u0026rsquo;s, I am sharing the discovered recipe to achieve what I needed!\nSystem\u0026rsquo;s info #====================================== Product Info ====================================== GPU[0] : Card Series: AMD Radeon RX 7900 XT GPU[0] : Card Model: 0x744c GPU[0] : Card Vendor: Advanced Micro Devices, Inc. [AMD/ATI] GPU[0] : Card SKU: D70401XT GPU[0] : Subsystem ID: 0x471e GPU[0] : Device Rev: 0xcc GPU[0] : Node ID: 1 GPU[0] : GUID: 37920 GPU[0] : GFX Version: gfx1100 ============================== Version of System Component =============================== Driver version: 6.19.6-arch1-1 and the jax version\nJAX version = 0.8.2 Running the docker container #Create the Dockerfile and build it. The Dockerfile in question, that includes the basic python packages needed for the work. Note again the usage of marimo, my default notebook, which I use for my experiments.\nFROM rocm/jax:rocm7.2.4-jax0.8.2-py3.12 WORKDIR /workspace RUN python3 -m pip install --upgrade --ignore-installed pip \u0026amp;\u0026amp; \\ python3 -m pip install \\ \u0026#34;marimo[recommended]\u0026#34; \\ \u0026#34;pymc\u0026gt;=6\u0026#34; \\ \u0026#34;jax==0.8.2\u0026#34; \\ \u0026#34;jaxlib==0.8.2+rocm7.2.4\u0026#34; \\ numpyro \\ blackjax \\ arviz \\ pandas \\ polars \\ matplotlib \\ scipy \\ scikit-learn \\ bokeh \\ holoviews \\ hvplot \\ ipykernel \\ xarray \\ netCDF4 EXPOSE 2718 CMD [\u0026#34;bash\u0026#34;, \u0026#34;-lc\u0026#34;, \u0026#34;exec marimo edit --headless --host 0.0.0.0 --port 2718 /workspace/notebook.py\u0026#34;] A script, created and adjusted with a help of an LLM, to facilitate the daily run of the container and facilitate the usage:\n#!/usr/bin/env bash set -e IMAGE_NAME=marimo-amd-jax CONTAINER_NAME=marimo-amd-jax PORT=2718 docker build -t \u0026#34;$IMAGE_NAME\u0026#34; . docker run --rm -it \\ --name \u0026#34;$CONTAINER_NAME\u0026#34; \\ --device=/dev/kfd \\ --device=/dev/dri \\ --group-add video \\ -p $PORT:$PORT \\ -v \u0026#34;$PWD\u0026#34;:/workspace \\ -w /workspace \\ \u0026#34;$IMAGE_NAME\u0026#34; \\ marimo edit --headless --host 0.0.0.0 --port $PORT notebook.py Inside the container #Within the container, we run python scripts/notebooks as usual. For my case, a necessary variables must be defined for a functional sampling.\nBefore jax is imported, some variables must be exported:\nimport os os.environ[\u0026#34;XLA_FLAGS\u0026#34;] = \u0026#34;--xla_gpu_enable_command_buffer=\u0026#39;\u0026#39;\u0026#34; os.environ[\u0026#34;HIP_VISIBLE_DEVICES\u0026#34;] = \u0026#34;0\u0026#34; os.environ[\u0026#34;ROCR_VISIBLE_DEVICES\u0026#34;] = \u0026#34;0\u0026#34; #os.environ[\u0026#34;XLA_PYTHON_CLIENT_PREALLOCATE\u0026#34;] = \u0026#34;false\u0026#34; os.environ[\u0026#34;XLA_PYTHON_CLIENT_PREALLOCATE\u0026#34;] = \u0026#34;true\u0026#34; os.environ[\u0026#34;XLA_PYTHON_CLIENT_MEM_FRACTION\u0026#34;] = \u0026#34;0.95\u0026#34; os.environ[\u0026#34;JAX_TRACEBACK_FILTERING\u0026#34;] = \u0026#34;off\u0026#34; os.environ[\u0026#34;JAX_ENABLE_X64\u0026#34;] = \u0026#34;False\u0026#34; os.environ[\u0026#34;PYTENSOR_FLAGS\u0026#34;] = \u0026#34;floatX=float32\u0026#34; ### import jax import pytensor pytensor.config.floatX = \u0026#34;float32\u0026#34; In order to sample using the jax sampler, we implicitly import the jax sampler\u0026rsquo;s functionality. The first option is to use the numpyro\u0026rsquo;s sampler:\nfrom pymc.sampling.jax import sample_jax_nuts ### idata = sample_jax_nuts( model=pymc_model, draws=1000, tune=1000, chains=4, nuts_sampler=\u0026#39;numpyro\u0026#39;, chain_method=\u0026#34;vectorized\u0026#34;, progressbar=True, random_seed=42, ) A second option is the blackjax sampler, which still experiences issues with IO, parallelism. The working configuration with a progress tracker is the following:\nidata = sample_jax_nuts( model=pymc_model, draws=1000, tune=1000, chains=1, ## note here nuts_sampler=\u0026#39;blackjax\u0026#39;, chain_method=\u0026#34;vectorized\u0026#34;, #chain_method=\u0026#34;parallel\u0026#34;, progressbar=True, random_seed=42, ) it is possible, however, to increase the number of chains, which is solved by setting chain_method=\u0026quot;vectorized\u0026quot;, and higeher number of chains.\n","date":"July 11, 2026","permalink":"https://blog.leokrglv.net/notes/jax_sampler_amd/","section":"Notes","summary":"\u003cp\u003eThis is a small note on how to use a GPU-based sampler on linux with an AMD GPU.\u003c/p\u003e\n\u003cp\u003eWhen using \u003ccode\u003ePyMC\u003c/code\u003e, it is possible to use external samplers, that increase the sampling speed.\nFor CPU-based samplers, the best-known choice is the Rust-based \u003ccode\u003enutpie\u003c/code\u003e.\u003c/p\u003e\n\u003cp\u003eIf the underlying model has a high number of levels and/or any type of significant hierarchies, one may want to consider a GPU-based sampler.\nFor that, the common solutions are jax-based \u003ccode\u003enumpyro\u003c/code\u003e and \u003ccode\u003eblackjax\u003c/code\u003e samplers. To properly perform them, we need a dedicated GPU.\nAs of today, the most solid ecosystem, even on Linux, is, of course, the Nvidia\u0026rsquo;s CUDA. So the most frictionless way to make the common libraries is using the CUDA infrastructure.\nI am, currently, an AMD user, so making common libraries work to use GPU\u0026rsquo;s capability, is often not trivial.\u003c/p\u003e","title":"jax-based samplers on AMD's ROCm - a recipe"},{"content":"This is the short-form section of the blog.\nUse it for compact notes, tiny tutorials, quick reminders, and small tricks that do not need a full post.\nIf you want a more formal write-up, use posts. If you want code-heavy or project-sized content, use projects.\n","date":null,"permalink":"https://blog.leokrglv.net/notes/","section":"Notes","summary":"\u003cp\u003eThis is the short-form section of the blog.\u003c/p\u003e\n\u003cp\u003eUse it for compact notes, tiny tutorials, quick reminders, and small tricks that do not need a full post.\u003c/p\u003e\n\u003cp\u003eIf you want a more formal write-up, use \n      \n    \u003ca href=\"https://blog.leokrglv.net/posts/\"\u003eposts\u003c/a\u003e. If you want code-heavy or project-sized content, use \n      \n    \u003ca href=\"https://blog.leokrglv.net/projects/\"\u003eprojects\u003c/a\u003e.\u003c/p\u003e","title":"Notes"},{"content":"The Problem #Quality control and testing actions are always mandatory before delivering a product or a service. In this particular case study, we are considering a some manufacturing data recording during testing. Every row is the result of a testing procedure. The tabular data may look like this\ntimestamp id result retests 13:32:10 #AA3 pass 1 13:32:20 #AA4 failmode1 1 13:32:30 #AA5 pass 1 13:32:50 #AA4 failmode1 2 13:33:20 #AA4 pass 3 From this tabular data, we may look how many units passed on first try, how much units failed with mode 2 on first, second, \u0026hellip; tries etc\u0026hellip; When a unit fails some $K$ times, it is scrapped. In our demo, we take $K$ to be 4. Different failure modes depend on a various factors. We may divide this into two main \u0026ldquo;categories\u0026rdquo; - the testing procedure or the intrinsic problem of the unit. The examples below use synthetic data generated \u0026ldquo;by hand\u0026rdquo; with the same structure as the testing data I want to discuss.\nThe dataset may be modeled in various, sometimes extremely complicated ways, which we will keep for later discussions, to model both the temporal and spacial dependences. For now, we focus on simple summary statistics and fit different types of models, since sometimes, we all need simple things! Another reason of using a summary table are the significant costs related to work/retrieve the full data.\nThe goal is to describe the summary statistics using some models and observe the severity of some failure modes. Namely, if some failure mode is intrinsic to the batch/units due to some manufacturing issues, many units will be failing with this mode without \u0026ldquo;recovering\u0026rdquo;. We will thus want to somehow quantify this issue.\nThe summary table will thus look like this\nfailure_mode rank count SUCCESS 2 29769 FMODE_C 2 351 FMODE_C 3 201 FMODE_C 1 1038 We can illustrate the difference between a \u0026ldquo;real\u0026rdquo; problem and a \u0026ldquo;random\u0026rdquo; one.\nThe evolution of number of errors/retests (retesting a unit means it didn't pass the test during the previous test). The mode B _decays_ quickly, but not the mode D, suggesting the latter is worth looking into. First-order estimation #As we are working with a short summary table, it is clear we are losing a lot of information and relationships. Indeed, we dropped all of the spatio-temporal dependence. Our first central assumption consists of considering that there exists a unique probability for each failure mode. Namely, $$ \\mathbb{P}(\\text{fail} \\in (M,r) | \\text{fail} \\in (M, r-1)) = \\mathbb{P}(\\text{fail} \\in (M,r)) $$ , which is almost always not true due to the persistence of a failure mode, due to the temporal dependence and many other reasons. Yet this assumption is a good starting point, especially for discovering and learning about the processes.\nOur assumption assumes a model, where the number of units getting $r$-retested will decay geometrically. Let $$ C_{M,r} $$ the number of testing counts at failure mode $M$ and rank $r$. Our first-order decay model gives $$ C_{M,r} \\sim C_{M, r-1}q_{M} $$ Where $q_M$ is what we will call here the persistence factor of the mode $M$. This characterizes how good the tested units are \u0026ldquo;recovered\u0026rdquo; after failures. We may write for any testing attempt $$ C_{M,r} \\sim C_{M,1} q_{M}^{r-1} $$A small persistence would mean that after the first retest, there will be few units to be retested more, so the decay will be quick.\nThe first estimate will therefore be by simply computing the ratio $$ \\hat{q} \\coloneqq (\\frac{C_{M,K}}{C_{M,1}}) ^{1/(K-1)} $$ which is a great first estimate for what we need - to quantify the persistence effect of a certain mode.\nPoisson model #The Poisson model is a well-known model that models events or count data. This is the reason we will be interested in using it to model our situation. In fact, in our setup, there are important notions of success/failure, number of re-tests, \u0026hellip; Intuitively, this signals the usage of Bernoulli, geometric and other distributions. In the first order, however, we will consider the retest ranks simply as binned/histogram events.\nRemember our discrete decaying model. In this model, we expect the number of retests $\\mu$ to behave as $$\\mu_{M,r} \\sim A_{M} q_{M}^{r-1}$$It is highly tempting to use some $\\log$ transformation here, so taking it from both sides and developing gives $$ \\log[ \\mu_{M,r}] = \\log[ A_{M} q_{M}^{r-1} ] \\\\ \\log[ \\mu_{M,r}] = \\log[A_M] + (r-1)\\log[ q_M ] $$where we rename the quantities $\\log q \\mapsto \\alpha $ and $\\log A \\mapsto \\beta$, and obtain the expression $$ \\log(\\mu_{m,r}) = \\alpha_m + \\beta_m (r-1) $$which looks exactly the same as the Poisson regression written in the GLM formalism!\nFrequentist fit #Before proceeding with a (arguably) more complete version of a Bayesian model, we will fit the Poisson GLM. The result for one of the decay modes can be illustrated as below\nAfter fitting a Poisson model, there are multiple ways of assessing the quality of the fitting method/fit. One of the most common things is the post-hoc check the Poissonian equidispersion assumption, which is the known property of variance being equal to the mean.\nWe say that the data is overdispersed (the most common case in real-world data) when the variance of the data - how the observed data deviates from the prediction/mean is not of the order of the predicted mean (Var=Mean).\nThe most common way to do that is to use Pearson\u0026rsquo;s residuals 1. The Pearson\u0026rsquo;s residual is defined as $$ r_i^P = \\frac{y_i - \\hat{\\mu}_i}{\\sqrt{ \\hat{\\mu}_i }} $$The variance of the residual can be obtained by taking the variance of $y_i$, which is $\\mu_i$ (Poissonian property), and the denominator gives a $1/\\mu$ scaling, giving that $\\text{Var}(r_i^P) \\approx 1$. Same for the expectation of $r_i^P$. We can then sum the squares of $(r_i^P)^2$ and sum over $i$\u0026rsquo;s, and it turns out (known properties of the Pearson\u0026rsquo;s residuals) that the sum denoted as Pearson\u0026rsquo;s statistic $X_P^2$ has a mean of $n-p$ under the equidispersion assumption. Thus, if the ratio $$ X_P^2 /(n-p) $$ is strongly higher/lower than $1$, the data is over/under dispersed. In our case, the data is overdispersed, which is expected for a read industrial data. The most straightforward solution for that would be to use the quasi-Poissonian family, which will \u0026ldquo;artificially inflate\u0026rdquo; the variance or to use the negative Binomial distribution to fit the data.\nBayesian approach #After a simple statsmodels model definition and fit, we will use the Bayesian approach to fit the same model. (as it is known to be much more intuitive and slightly scientifically superior).\nSince the models implies a different Poisson regression for each failure mode $m$, this technically becomes a multilevel model. The model for every dataframe record $i$ is given by $$ C_i \\sim \\text{Pois}(\\mu_i) \\\\ \\log(\\mu_{i}) = \\alpha_{m[i]} + \\beta_{m[i]}(r-1) $$ where $C_i$ - the count we\u0026rsquo;re trying to model. We sample the model and we obtain the corresponding coefficients. The decay rate is obtained through $q_m = e^{\\beta_m}$.\nTwo persistence rates $q_m$ for $m=$FMODE_A $m=$FMODE_C (blue and red respectively). When inspecting the parameter distributions, it is natural to assess how likely it is that one of the failure mode is more/less persistent than the other. Within the Bayesian framework one is naturally led to compute it directly from distributions as $q_1 - q_2$ and compare it to $0$.\nAs mentionned before, after fitting a Poisson-like model, it is handy to appraise the overdispersion, which is again nicely computed using full distributions. We compare the deviation of predicted vs observed and predicted vs generated by the model. The two statistics are $T_\\text{obs}$ and $T_\\text{ppc}$ (posterior predictive check). A large ratio of obs to ppc signifies that the observed variance is greater than the one predicted by the model.\nBayesian showcase of overdispersed Poissonian model. Next steps #Based on evidence on overdispersion, we may consider an alternative, for example the negative binomial model. In our case, we will proceed with a slightly more involved dataset, that will be the base to a more informative geometric model.\nThe geometric model #We now return to the previous dataset, that has the form\ntimestamp id result 13:32:10 #AA3 pass 13:32:20 #AA4 failmode1 13:32:30 #AA5 pass Let\u0026rsquo;s recall the geometric model! The question it models \u0026ldquo;the number of trials needed to achieve the first success in a series of independent Bernoulli trials\u0026rdquo;, or simply put, \u0026ldquo;the number of attempts before a success\u0026rdquo;. Clearly, this question feels somewhat similar to our problem. Note that we clearly oversimplify our problem - by e.g. assuming that the trials are independent.\nLet\u0026rsquo;s define the parameters and set up the problem. For a unit with id $i$, let\u0026rsquo;s define $F_i$ as the $\\text{number of failed attempts before success or scrap}$. Our end goal would be to esimate the probability $p$, that we assume to be dependent on the mode. So we define $p_{m_i}$ - the probability of pass given the unit $i$ that belongs to mode $m$. There are multiple ways of defining the mode $m$, that the unit $i$ belongs to. Here, we will define as the first failure mode.\nFor example, for a given unit belonging to mode $m$, $$ \\mathbb{P}(F=0) = p_m \\newline \\mathbb{P}(F=1) = p_m(1-p_m)\\newline \\mathbb{P}(F=2) = p_m(1-p_m)^2 \\newline ... $$Given that we have a maximum retest number $K$, the likelihood can be written as\n$$ L_i (p_ {m_i}) = p_{m_i}^{s_i} (1 - p_{m_i} )^{f_i} $$ where we define $f_i = \\text{ the number of failed attempts}$ and $$ s_i= \\begin{cases} 1 \u0026 \\text{ passed} \\\\\\\\ 0 \u0026 \\text{ failed more than } K \\text{ times} \\end{cases} $$and the total likelihood by mode $$ L_m(p_m) = \\prod_{i: m_i \\in m} p_{m_i}^{s_i} (1 - p_{m_i} )^{f_i} $$ from which we define the total number of success belonging to the mode $$ S_m = \\sum_{i: m_i \\in m } s_{m_i} $$ and the total number of failed attempts as $$ F_m = \\sum_{i: m_i \\in m } f_{m_i} $$from which the log-likelihood is given by $$ \\text{loglik}_m = S_m \\log(p_m) + F_m \\log( 1- p_m) $$Finding the maximum gives the maximum likelihood estimator of $p_m$ as $$ \\hat{p}_m = \\frac{S_m}{F_m+S_m} $$Short note: In order to fit the probabilities of a given failure mode, we have shown that a natural distribution is the Geometrical one. We have mentioned that it follows from sequential Bernoulli trials. Therefore we may even fit the Bernoulli distribution directly without passing through the Geometrical distribution.\nWe model our Geometrical distribution for bayesian inference as\n$$ \\ell_i = s_i \\log(p_{m_i}) + f_{m_i} \\log( 1- p_{m_i}) $$ where we define the prior for $p$ as $p_m \\sim \\text{logit} (\\mathcal{N}(0,1))$. Here, as opposed to a \u0026ldquo;usual\u0026rdquo; model specification, we have an explicit log-likelihood expression, which will be defined through the PyMC\u0026rsquo;s Potential() functionality.\nBefore fitting, we will slightly aggregate the data - we take only unique ID\u0026rsquo;s of the units and the computed values $s_i$, $f_i$ and the assigned mode $m_i$. Within the model, we estimate the per-mode probability\nEstimated parameter for probability of one mode. Comparing the two baseline models #Our problem of test/re-test has been evaluated/fitted using two models. It is clear that the underlying process of the related random variables are different.\nThe poisson distribution describes the number of events that will occur at some known rate. And in this case, we took the event to be the retest. Within these assumptions the events are independent and have constant rates (which are the assumptions) we are willing to accept.\nThe geometric distribution is based on the Bernoulli trials and answers a question of \u0026ldquo;how many trials until a success\u0026rdquo;. Although we know that \u0026ldquo;retesting\u0026rdquo; and event \u0026ldquo;trial until success\u0026rdquo; are related, we are still asking a different question.\nWe can plot the estimated parameters of the two methods side-by-side.\nComparing estimators - Poisson vs Geometrical models. We see that the estimators are different for some of the failure modes. We see that the Poisson model probabilities are (almost) always inferior to the Geometric ones. The differences are sometimes considerable. The FMODE_D, however, is similar using both models, which was the initial goal of the problem - to spot significantly persistent modes! To attempt to justify the differences we must consider each of them separately.\nThe success mode, clearly, does not make sense to be estimated with the geometric distribution. Indeed, we recall that the question is \u0026ldquo;what is the expected retries before success given the unit belongs to a failure mode $m$\u0026rdquo;. Where $m$ was defined as the first mode the unit occurred with. So if a unit belongs to the SUCCESS mode, we do not expect to have re-tries.\nThe failure mode D is the mode that was intended to be caught as the persistent mode, and both models highly agree with a high value of $p_{m}$.\nFailure modes A (B,C). The Poisson model estimates how the count of a mode changes across retest ranks. It does not necessarily know whether the same units are carrying the same mode across time. A unit can start with FMODE_A, then later fail with FMODE_B or FMODE_C. In that case, the histogram count of FMODE_A may drop quickly, even if units that initially had FMODE_A continue to fail under different labels.\nThe geometric model, on the other hand, assigns each unit to a single mode, such as the first observed failure mode. It then asks whether units in that group keep failing. Therefore, a mode can have low Poisson persistence but high geometric persistence if the failure label disappears from the histogram while the units themselves continue failing under other modes. This is the case, where the unit is actually defect and is flagged by belonging to some mode.\nThe Interactive notebook #Since the notebook\u0026rsquo;s functionality relies heavily on PyMC and large data, it is problematic to run the marimo notebook in WASM embedded environment. For this reason, the notebook is directly available on my github, where a single notebook is available.\nHere, no formal proof or explanation is provided. Instead a handwavy heuristics to be able to feel the result.\u0026#160;\u0026#x21a9;\u0026#xfe0e;\n","date":"June 25, 2026","permalink":"https://blog.leokrglv.net/posts/failure_retest/","section":"Posts","summary":"\u003ch2 id=\"the-problem\" class=\"relative group\"\u003eThe Problem \u003cspan class=\"absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100\"\u003e\u003ca class=\"group-hover:text-primary-300 dark:group-hover:text-neutral-700\" style=\"text-decoration-line: none !important;\" href=\"#the-problem\" aria-label=\"Anchor\"\u003e#\u003c/a\u003e\u003c/span\u003e\u003c/h2\u003e\u003cp\u003eQuality control and testing actions are always mandatory before delivering a product or a service.\nIn this particular case study, we are considering a some manufacturing data recording during testing.\nEvery row is the result of a testing procedure. The tabular data may look like this\u003c/p\u003e\n\u003ctable\u003e\n  \u003cthead\u003e\n      \u003ctr\u003e\n          \u003cth\u003etimestamp\u003c/th\u003e\n          \u003cth style=\"text-align: right\"\u003eid\u003c/th\u003e\n          \u003cth style=\"text-align: right\"\u003eresult\u003c/th\u003e\n          \u003cth style=\"text-align: right\"\u003eretests\u003c/th\u003e\n      \u003c/tr\u003e\n  \u003c/thead\u003e\n  \u003ctbody\u003e\n      \u003ctr\u003e\n          \u003ctd\u003e13:32:10\u003c/td\u003e\n          \u003ctd style=\"text-align: right\"\u003e#AA3\u003c/td\u003e\n          \u003ctd style=\"text-align: right\"\u003epass\u003c/td\u003e\n          \u003ctd style=\"text-align: right\"\u003e1\u003c/td\u003e\n      \u003c/tr\u003e\n      \u003ctr\u003e\n          \u003ctd\u003e13:32:20\u003c/td\u003e\n          \u003ctd style=\"text-align: right\"\u003e#AA4\u003c/td\u003e\n          \u003ctd style=\"text-align: right\"\u003efailmode1\u003c/td\u003e\n          \u003ctd style=\"text-align: right\"\u003e1\u003c/td\u003e\n      \u003c/tr\u003e\n      \u003ctr\u003e\n          \u003ctd\u003e13:32:30\u003c/td\u003e\n          \u003ctd style=\"text-align: right\"\u003e#AA5\u003c/td\u003e\n          \u003ctd style=\"text-align: right\"\u003epass\u003c/td\u003e\n          \u003ctd style=\"text-align: right\"\u003e1\u003c/td\u003e\n      \u003c/tr\u003e\n      \u003ctr\u003e\n          \u003ctd\u003e13:32:50\u003c/td\u003e\n          \u003ctd style=\"text-align: right\"\u003e#AA4\u003c/td\u003e\n          \u003ctd style=\"text-align: right\"\u003efailmode1\u003c/td\u003e\n          \u003ctd style=\"text-align: right\"\u003e2\u003c/td\u003e\n      \u003c/tr\u003e\n      \u003ctr\u003e\n          \u003ctd\u003e13:33:20\u003c/td\u003e\n          \u003ctd style=\"text-align: right\"\u003e#AA4\u003c/td\u003e\n          \u003ctd style=\"text-align: right\"\u003epass\u003c/td\u003e\n          \u003ctd style=\"text-align: right\"\u003e3\u003c/td\u003e\n      \u003c/tr\u003e\n  \u003c/tbody\u003e\n\u003c/table\u003e\n\u003cp\u003eFrom this tabular data, we may look how many units passed on first try, how much units failed with mode 2 on\nfirst, second, \u0026hellip; tries etc\u0026hellip; When a unit fails some $K$ times, it is scrapped.\nIn our demo, we take $K$ to be 4.\nDifferent failure modes depend on a various factors. We may divide this into two main \u0026ldquo;categories\u0026rdquo; - the testing procedure or the intrinsic problem of the unit.\nThe examples below use synthetic data generated \u0026ldquo;by hand\u0026rdquo; with the same structure as the testing data I want to discuss.\u003c/p\u003e","title":"Poisson, Geometric \u0026 Survival Models: A Hazard-Based Look at Persistent Failure in Retesting"},{"content":"This page collects the code-oriented work behind the blog: experiments, notebooks, and small implementations that connect statistical ideas with working Python.\nSilverman\u0026rsquo;s test for multimodality #An implementation-oriented look at Silverman\u0026rsquo;s test for detecting modes in a distribution. The project covers KDE bandwidth selection, bootstrap p-values, variance correction, and the practical limits of using classical tests for shape recognition.\nRead the post Open the interactive notebook View the repository CUDA notes #Personal notes on CUDA programming, including synchronization patterns, cooperative groups, and warp-level primitives. The write-up collects material from CUDA documentation, tutorials, and books into one practical reference.\nRead the post Open the PDF ","date":"May 31, 2026","permalink":"https://blog.leokrglv.net/projects/","section":"Home","summary":"\u003cp\u003eThis page collects the code-oriented work behind the blog: experiments, notebooks, and small implementations that connect statistical ideas with working Python.\u003c/p\u003e\n\u003ch2 id=\"silvermans-test-for-multimodality\" class=\"relative group\"\u003eSilverman\u0026rsquo;s test for multimodality \u003cspan class=\"absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100\"\u003e\u003ca class=\"group-hover:text-primary-300 dark:group-hover:text-neutral-700\" style=\"text-decoration-line: none !important;\" href=\"#silvermans-test-for-multimodality\" aria-label=\"Anchor\"\u003e#\u003c/a\u003e\u003c/span\u003e\u003c/h2\u003e\u003cp\u003eAn implementation-oriented look at Silverman\u0026rsquo;s test for detecting modes in a distribution. The project covers KDE bandwidth selection, bootstrap p-values, variance correction, and the practical limits of using classical tests for shape recognition.\u003c/p\u003e\n\u003cul\u003e\n\u003cli\u003e\n      \n    \u003ca href=\"https://blog.leokrglv.net/posts/silverman/\"\u003eRead the post\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"/notebooks/silverman/\"\u003eOpen the interactive notebook\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"https://github.com/leokruglikov/silverman_statistical_test\" target=\"_blank\" rel=\"noreferrer\"\u003eView the repository\u003c/a\u003e\u003c/li\u003e\n\u003c/ul\u003e\n\u003ch2 id=\"cuda-notes\" class=\"relative group\"\u003eCUDA notes \u003cspan class=\"absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100\"\u003e\u003ca class=\"group-hover:text-primary-300 dark:group-hover:text-neutral-700\" style=\"text-decoration-line: none !important;\" href=\"#cuda-notes\" aria-label=\"Anchor\"\u003e#\u003c/a\u003e\u003c/span\u003e\u003c/h2\u003e\u003cp\u003ePersonal notes on CUDA programming, including synchronization patterns, cooperative groups, and warp-level primitives. The write-up collects material from CUDA documentation, tutorials, and books into one practical reference.\u003c/p\u003e","title":"Projects"},{"content":"The Problem #Consider an industrial set of measurements, like a parameter from a batch of products. If we assume that the realisations of our measured random variable are IID, we expect that the resulting distribution is gaussian. However, if the underlying process has genuinely multiple sources, the resulting distribution will be a mixture of gaussians, or a mixture of other distributions. Such process may occur if the there is a significant difference in the originating raw material batches.\nThis is the reason, why it is important to be able to identify a potential bi/multi-modality in the observed distribution. Arguably, the best method would be to visually inspect all of the distributions, which is sometimes not possible. A visual inspection can be replaced by a shape recognition algorithm or similar. The resources to run such an algorithm may be limited or unavailable due to other reasons. This is why one may want to rely on \u0026ldquo;older\u0026rdquo;/classical methods such as the usual hypothesis tests.\nThere are multiple statistical tools and tests to verify multimodality. The most common ones are the Hardigan\u0026rsquo;s dip test and the Silverman\u0026rsquo;s test. We will look and implement the latter one.\nThe core idea #The Silverman\u0026rsquo;s hypothesis tests whether the underlying distribution has $k$ or less modes ($H_0$) or more than $k$ modes ($H_1$). The main idea consists of two big steps\nUsing KDE, smooth out the data using the $h_\\text{crit.}$, where it is the threshold, at which the KDE becomes exactly $k$-modal Perform the bootstrap using this bandwidth. If many of the samples yield a higher-modal distribution, there is more evidence to reject the $H_0$. The Kernel Density Estimation #In Silverman\u0026rsquo;s test, the KDE method is extensively used. In practice, this means smoothing out our histogram/data. Intuitively, this means approximating every \u0026ldquo;peak\u0026rdquo; of our histogram by a Gaussian \u0026ldquo;hat\u0026rdquo; and summing over all the range. Mathematically, this means to create a new function $\\hat{f}_h(x)$ defined by $$ \\hat{f}(x) = \\frac{1}{Nh}\\sum_i^N K \\Bigl( \\frac{x- x_i}{h} \\Bigr) $$ ,where $N$ is the sample size of the dataset.\nThe critical bandwidth #The core parameter of the KDE is the bandwidth, which determines how wide is the \u0026ldquo;hat\u0026rdquo; we\u0026rsquo;re fitting to each data point. As a result, the higher is the bandwidth, the more smoothing we observe. A low $h$ means the KDE would \u0026ldquo;react\u0026rdquo; to every noise bump, resulting in a function with many local maxima/bumps. On the other hand, if the bandwidth is too wide, the histogram smoothing will be too smooth and will not capture all of the modes of our distirbution.\nGiven the parameter of the null hypothesis $k$, the first thing to do is to the critical bandwidth, so that it is \u0026ldquo;barely\u0026rdquo; giving a $k$-mode distribution. Mathematically, we write it as $$ h_\\text{crit} = \\text{inf} ( h: \\hat{f}_h \\text{ has } k \\text{ modes or less} ) $$That is, this $h_\\text{crit}$ is just at the edge - if we reduce it even slightly, the number of modes will increase from $k$. Perfectly, the number of modes of $\\hat{f}$ with $h_\\text{crit}$ is $k$ and if we add a tiny \u0026ldquo;perturbation\u0026rdquo; $h_\\text{crit}+\\epsilon$, the number of modes becomes $k+1$.\nThe search of the $h_\\text{crit}$ is a binary search:\nStart with a big $h$ If the number $k$ of the target modes is larger, we reduce it by half If the new number of the modes is smaller than $k$, reduce it by half. If larger, multiply it by 2. Repeat the steps until we obtain the $h_\\text{crit}$. The number of peaks can be obtained in different ways, but we use the default find_peaks() method from scipy.\nWe can show how the number of modes changes with the critical bandwidth.\nFigure: Histogram of bootstrapped parameters illustrating uncertainty. The bootstrap #The second step is the bootstrap step, which inherently uses the previously found $h_\\text{crit}$. Given the original data $[x_1, x_2, ..., x_N]$, the algorithms can be summarized as follows:\nGenerate a sample set from the original data with replacement $[x_{J_1}, x_{J_2}, ..., x_{J_n}]$, where $\\{J_{1}, J_{2}, ..., J_{n}\\}$ a set of randomly generated indices with replacement. Add a scaled gaussian noise to each point: $h_\\text{crit}z$ with $z\\sim \\mathcal{N}(0,1)$ to obtain the bootstrapped dataset $x^{\\ast}$, $\\{x_1, x_2^\\ast, ... , x_n^\\ast\\}$ where $x_i^\\ast \\coloneqq x_{J_i} + h_\\text{crit}z$. We can compute the critical bandwidth based on this newly created dataset $\\{x_i^\\ast\\}$ denoted $h_\\text{crit}^*$. Repeat the previous steps $B$ times to obtain a bootstrapped set of bandwidth $\\{ h_{\\text{crit}, i}^{*}\\}$ with $i \\in [1,...,B]$. Compute the $p$-value as the number of times the critical bandwidth was larger than the one \u0026ldquo;original\u0026rdquo; $h_\\text{crit}$ divided by $B$. Mathematically $$p = \\frac{1}{B} \\sum_i^B I(h_{\\text{crit},i} \u003e h_\\text{crit})$$ where $I$ - the indicator function. The intuition is the following: if the actual distribution really has more than k peaks, the observed critical bandwidth $h_\\text{crit}$ will tend to be large, because substantial smoothing is needed before the KDE has max. k modes. Therefore, under $H_0$, such a large $h_\\text{crit}$ is unlikely, leading to a low $p$-value and rejection of $H_0$.\nWorking example with marimo # Corrections #The Silverman\u0026rsquo;s test is known to be conservative. That means that there is a chance to fail to accept the $H_1$ (more precisely fail to reject $H_0$). In other words, it is bad at detecting modes if there are actually more than k modes, if these modes are poorly separated or have low weight.\nVariance correction #The first correction will be reffered to as the \u0026ldquo;variance correction\u0026rdquo;. This correction involves slightly changing the generation of the $[x^*_i]$. The correction scales the generated data to reduce the variance.\nThe variance of the full quantity $$ \\text{Var}(x^*) = \\text{Var}(x) + \\text{Var}(hz) = \\sigma^2 + h^2 $$ where $\\sigma^2$ - the variance of the sample and $h$ - the critical width. Therefore the newly generated data is slightly inflated by $h$. Intuitively, the data is more spread out, meaning detecting new modes is harder. To mitigate the issue, we want to scale the second term: $$ x = \\bar{x} + \\frac{x+hz-\\bar{x}}{\\sqrt{1+h^2/\\sigma^2}} $$ which gives the same total variance $\\sigma^2$.\nAsymptotic correction #The second correction is known as the Hall-York correction and consists in comparing the bootstrapped $h$ to a scaled $h_\\text{crit}$. That is, the $p$-value is computed as $$ p = \\frac{1}{B} \\sum_i^B I(h_{\\text{crit},i} \u003e \\lambda h_\\text{crit}) $$This scaling factor is what helps us to deal with the conservativeness. The basic idea that motivates this in the paper is that the critical bandwidth assymptotically differ. Namely, $$\\frac{ h_\\text{crit}^*}{h_\\text{crit}} \\neq 1$$ even asymptotically. In order to find the $\\lambda$, we can empirically track this discrepancy using Monte Carlo. We can actually use the value from the paper and use the fit format to compute $\\lambda$, that depends on our confidence level $\\alpha$. A typical value, that we use is around $0.8$.\nTesting #Simple case #Let\u0026rsquo;s first look at how the Silverman\u0026rsquo;s test works with the simplest case. As the simplest case, we take the number of modes in the $H_0$ to be $k=2$ and generate a bimodal distribution.\nThe p-value distribution is significantly larger than any $\\alpha$ threshold in almost any simulation and the test gives a sure value.\nMore exotic cases #It is possible to look at how the tests behave at different regimes in the Silverman\u0026rsquo;s test repo, where multiple notebooks for experimenting can be found.\nConclusion #The goal of the article is to introduce to the notion of the Silverman\u0026rsquo;s statistical test. We showed the main theoretical concepts and potential room for improvement. In order to experiment with the different data and with different test variants, all of the resources are available in Silverman\u0026rsquo;s test repo.\n","date":"May 4, 2026","permalink":"https://blog.leokrglv.net/posts/silverman/","section":"Posts","summary":"\u003ch1 id=\"the-problem\" class=\"relative group\"\u003eThe Problem \u003cspan class=\"absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100\"\u003e\u003ca class=\"group-hover:text-primary-300 dark:group-hover:text-neutral-700\" style=\"text-decoration-line: none !important;\" href=\"#the-problem\" aria-label=\"Anchor\"\u003e#\u003c/a\u003e\u003c/span\u003e\u003c/h1\u003e\u003cp\u003eConsider an industrial set of measurements, like a parameter from a batch of products. If we assume that\nthe realisations of our measured random variable are IID, we expect that the resulting distribution is gaussian.\nHowever, if the underlying process has genuinely multiple sources, the resulting distribution will be a mixture of gaussians, or a mixture of other distributions. Such process may occur if the there is a significant difference in the originating raw material\nbatches.\u003cbr\u003e\nThis is the reason, why it is important to be able to identify a potential bi/multi-modality in the observed distribution. Arguably, the\nbest method would be to visually inspect all of the distributions, which is sometimes not possible. A visual inspection can be\nreplaced by a shape recognition algorithm or similar. The resources to run such an algorithm may be limited or unavailable\ndue to other reasons. This is why one may want to rely on \u0026ldquo;older\u0026rdquo;/classical methods such as the usual hypothesis tests.\u003c/p\u003e","title":"Finding modes in multimodal distribution with Silverman"},{"content":"The Problem #Consider the following known industrial problem - we manufacture samples (possibly with different dimensions, characteristics, etc\u0026hellip;) which we are then testing/verifying. What we want then is to characterize the measurement processes, as it may have multiple sources of variability. Indeed, the measurement process can depend on the operator and appraisal (i.e. testing machine), the part itself and potentially some other things that we include in our observations. Additionally to the mentioned single factors, interactions could also be significant (e.g. some tester or operator is better off with some particular type of the part).\nIn what will follow, we consider two (three) potential sources of errors - the one coming from the tester (or operator).\nA possible design, where all of the appraisals test all of the parts is shown below. Such design in the field of Design of Experiments (DoE) is known as a fully crossed (full factorial) design. However, we will simply refer to it as crossed design.\nWe can extend the idea of two degrees of freedom to three and more, with, let\u0026rsquo;s say, different operators using different appraisals testing different parts. Sometimes these degrees of freedom do not represent \u0026ldquo;real entities\u0026rdquo; and instead time. For example, instead of parts being attributed to a tester (parts 1,2,3 being tested with appraisal #14), we test a system 10 times within a day, then repeat again after a week [^10]. Although it seems different, this setup brings potential variability at different factors (tester $\\rightarrow$ part, week $\\rightarrow$ day).\nThe gauge R\u0026amp;R #The gauge within the Measurement System Analysis (MSA) is referred to the measurement system of our appraisal acting on different parts. The central concept of the gauge analysis is quantifying and characterizing the variances of the factors1. The two main factors of interest are parts and appraisers. The R\u0026amp;R here stands for Repeatability and Reproducibility of the gauge. Those characteristics are nothing but metrics based on the obtained numbers and will be defined below.\nThe classical idea of the Gauge R\u0026amp;R MSA within the field of quality control and $6\\sigma$-discipline is to decompose the variability of the process into predefined components.\nVariance Decomposition #The variability decomposition that is used within the framework is given by $$ \\text{Var}(y) = \\sigma_A^2 + \\sigma_P^2 + \\sigma_{PA}^2 + \\sigma_e^2 $$ $\\sigma_e^2$ the \u0026ldquo;error\u0026rdquo; variance originating from repeated measurement/randomness. This is the part known as residual variability or Repeatability - the variation under repeated measurements. $\\sigma_A^2$ - the variability of the appraisal/measurement system (again, we call it the measurement system, but it may refer to the operator or any other concept related to the measurement system). $\\sigma_P^2$ - the part-to-part variability $\\sigma_{PA}^2$ - the appraisal-to-part variability. In other words, the variation of the part-to-appraisal interaction (how good/bad a appraisal operates with a certain part). The MSA approach defines other quantities and equips them with meaning:\n$\\text{Reproducibility} = \\sigma_A^2 + \\sigma_{PT}^2$ - the variability that is brought by the appraisal. One defines the Total Gauge R\u0026amp;R as the sum of the appraisal variability and random errors: $\\text{Total Gauge R\\\\\u0026R}=\\sigma_e^2 + \\sigma_A^2 + \\sigma_{AP}^2$. This describes the total uncertainty that does not originate from parts. As a result one concludes that the total variation is given by $\\text{Total Variation}= \\text{Var}(y)= \\text{Total Gauge R}\\\\\u0026{R} + \\text{Part-to-part}$ ANOVA #The next step is to estimate the defined variances. In order to find the variance components from the sampled data, we use the known unbiased estimator - the mean square error.\nThe repeatability variance term describing the overall error term and is given by the MSE unbiased estimator, computed as residuals from the mean of repeated measurements $\\bar{y}_{ij\\cdot}$: $$ \\sigma_e^2 \\simeq \\text{MSE}_e = \\frac{\\sum _{ijk} (y _{ijk} - \\bar{y} _{ij\\cdot})^2}{n_P n_A (n_r - 1)} $$The remaining estimators for different $\\sigma^2$ can be computed using the expressions for the mean squares\n$$ \\text{MSE}_e = \\sigma_e^2 \\newline \\text{MSE} _{PA} = \\sigma_e^2 + n _r \\sigma _{PA}^2 \\newline \\text{MSE} _{A} = \\sigma_e^2 + n _r \\sigma _{PA}^2 + n _P n _r \\sigma_A^2 \\newline \\text{MSE} _{P} = \\sigma_e^2 + n _r \\sigma _{PA}^2 + n _A n _r \\sigma_P^2 $$ from which, for example, we easily obtain the estimator for one of the interaction terms\n$$ \\sigma_{PA}^2 = \\frac{\\text{MSE} _{PA} - \\text{MSE} _{e}}{n _r} $$This is the approach we use within the ANOVA framework in order to estimate the variance terms- iteratively taking the differences between the mean squared errors and scaling them by the appropriate degrees of freedom.\nIn practice, estimates may become negative due to sampling variability (if e.g. $\\text{MSE} _{PA} \u003c \\text{MSE} _e$). This issue is known within the framework, and we will see that iterative, ANOVA-free methods which estimate the variances are not immune to negative-variance issues.\nInterpreting results #In additional to trivial variance components shown before, one defines the the central gauge R\u0026amp;R quantity $\\sigma _{\\text{GRR}}^2 = \\sigma_A^2 + \\sigma _{PA}^2 + \\sigma_e^2$, which is the sum of all of the variations without the part-to-part variation. As a result, one can write the full variance as the sum $$ \\sigma _\\text{total}^2 = \\sigma _P^2 + \\sigma _{\\text{GRR}}^2 $$To each variance, it is natural to define the standard deviation. For example, the repeatability stdev $$ \\sigma_e \\coloneqq \\sqrt{\\sigma_e^2} $$ or the total GRR deviation $$ \\sigma_\\text{GRR} = \\sqrt{ \\sigma_P + \\sigma_{PA} + \\sigma_e^2} $$ Therefore, the $\\sigma_\\text{GRR}$ represents the total variation that is arising when the appraiser is intervening. In other words, the standard deviation due to the measurement system.\nFor a process that is measured, one may usually define some sort of specification limits (otherwise, measurement process cannot be properly evaluated, as the outcome is not compared nor compared against anything). We want the process to be located within our upper and lower specification limits - USL and LSL.\nIn \u0026ldquo;typical\u0026rdquo; industrial setting, we want the usual process variation to be within a controlled interval, e.g. a $s\\sigma$ interval, with, e.g. $s=6$, which is the known origin of the $6$-$\\sigma$ methodology. In order to understand and interpret the data, we consider a relative quantity, that will call here GtT - the Gauge(R\u0026amp;R)-to-Tolerance ratio, which we define as $$ \\text{GtT}_s = s \\cdot \\frac{\\sigma _\\text{GRR}}{\\text{Tol}}$$where $\\text{Tol}$ - the tolerance defined as the specification range $\\text{Tol} = \\text{USL} - \\text{LSL}$\nThe quantity actually represents is how well the $6$-$\\sigma$ variation \u0026ldquo;sits\u0026rdquo; within the control range. In other words, what is the ratio, of the variability range to the total control range.\nOne attempts to illustrate the concept of GRR being contained in the tolerance in the image below.\nIn this case the quantity $\\text{GtT}$ is smaller the better, as we want small variability of our appraisal compared to our control range. In practice, we consider the ratio to be good if $\\text{GtT}\u003c0.1$ (or $10\\%$) and acceptable when $0.1 \u003c \\text{GtT} \u003c 0.3$.\nInstead of $\\sigma_\\text{GRR}$ in the numerator of our ratio, we could use another derived standard deviation from another variation contribution, e.g. $\\sigma_{P}$. The ratio in that case strongly depends on the nature of the experiment. That is, the parts may have intentionally come from the same batch. This means we expect from our measurement system to measure them as same. Similarly, it is possible to use differently produced samples, which would differently respond to measurements. Therefore, the ratio $6\\sigma_{\\text{P}}/\\text{Tol}$ may have very different interpretations based on different designs.\nThe Linear model #Up to now, in the discussed design of experiment, we decomposed the total variance into the (pre-)defined components (natural variation, part, appraisal, part-to-appraisal) using the ANOVA method.\nIt is well known that ANOVA is nothing but a special case of the (general) linear model. Without going into details, this means that our response measurement variable that we denote $y$ responds to the inputs linearly as $$ y_{ijk} = \\mu + P_i + A_j + PA_{ij} + \\epsilon_{ijk} $$ where $P_i$ - the effect of part $i$, $A_j$ - the effect of appraisal $j$, $PA_{ij}$ the interaction of part $i$ and appraisal $j$, $\\epsilon_{ijk}$ - the \u0026ldquo;random\u0026rdquo; error of the run and $\\mu$ - the overall mean2.\nThis linear model is easily estimated via ANOVA method (as a first approach, means and variances) with mean squared errors as variance estimates (see $\\text{MSE}$ definitions above).\nMixed models #The framework of mixed models is known to work well with nested, multilevel, tabular data. In this case, the data is slightly more restricted, but nevertheless exhibits some nestedness - parts and appraisals. Indeed, we may fit the mixed models\u0026rsquo; engine to our experimental data within the linear model $$ y_{ijk} = \\mu + P_i + A_j + PA_{ij} + \\epsilon_{ijk} $$ , where each effect is modeled as\n$P_i \\sim \\mathcal{N}(0, \\sigma_P^2)$ $A_j \\sim \\mathcal{N}(0, \\sigma_A^2)$ $PA _{ij} \\sim \\mathcal{N}(0, \\sigma^2 _{PA})$ $\\epsilon_{ijk} \\sim \\mathcal{N}(0, \\sigma^2 _{E})$ The formulation is very intuitive, as we expect (and we want) each of the effect to be centered at 0 (from the overall mean $\\mu$) and each have their own variance.\nWhat changes in the mixed models\u0026rsquo; formulation is the way we can estimate the coefficients $\\sigma$\u0026rsquo;s for every effect. In ANOVA, we would run variance estimations and obtain the coefficients as \u0026ldquo;byproducts\u0026rdquo;. The framework of Multilevel Models, however, is much broader (in the sense of the ability to model phenomena) and the variances $\\sigma$\u0026rsquo;s are part of the model to be estimated, usually computed using the REML method. This is another fundamental difference between the two approaches.\nTabular real-world data often comes with missing and unbalanced entry. This means that for some of the nested factor combination, there are less or often missing data. In our case, this would mean that there are some combinations of parts-appraisals have different numbers, few numbers or no entries at all. This, in contrast to ANOVA, is naturally handled and taken into account.\\ In practice, this means we can avoid doing a full-factorial/full-rotation for some of MSA studies, in order to get a first overview and understanding of our systems.\nAnother possible improvement would be the normality assumption. In fact, this assumption has been present in both methods the whole time. In the case of the \u0026ldquo;original\u0026rdquo; ANOVA, the assumption is assumed on all levels. For the considered framework of the Multilevel Models this is also the case. It is nevertheless possible to extend the model to the Multilevel General Linear Models.\nImplementation #The described situation can be gracefully modeled in R. Python, however, is less natural for that. It is however possible to use the known statsmodel, that implements a part of the multilevel models\u0026rsquo; functionality.\nWe will now quickly go over the structure of the implementation in python.\\ We have a dataframe that contains data from the measurements in the long format. For example, we\u0026rsquo;re measuring some quantity $V$ (e.g. voltage), with the parts and appraisals being specified in the respective columns in the pandas/polars dataframe.\nimport pandas as pd import statsmodels.formula.api as smf d[\u0026#34;appraisal\u0026#34;] = d[\u0026#34;appraisal\u0026#34;].astype(\u0026#34;category\u0026#34;) d[\u0026#34;part\u0026#34;] = d[\u0026#34;part\u0026#34;].astype(\u0026#34;category\u0026#34;) # create interaction feature d[\u0026#34;appraisal_part\u0026#34;] = ( d[\u0026#34;appraisal\u0026#34;].astype(str) + \u0026#34;:\u0026#34; + d[\u0026#34;part\u0026#34;].astype(str) ).astype(\u0026#34;category\u0026#34;) # a dummy group trick - a single constant d[\u0026#34;_grp\u0026#34;] = 1 The model definition using the statsmodels must explicitly specify the variance components\nmodel = smf.mixedlm( \u0026#34;y ~ 1\u0026#34;, data=d, groups=d[\u0026#34;_grp\u0026#34;], re_formula=\u0026#34;0\u0026#34;, vc_formula={ \u0026#34;part\u0026#34;: \u0026#34;0 + C(part)\u0026#34;, \u0026#34;appraisal\u0026#34;: \u0026#34;0 + C(appraisal)\u0026#34;, \u0026#34;appraisal_part\u0026#34;: \u0026#34;0 + C(appraisal_part)\u0026#34;, }, ) res = model.fit(reml=True, method=\u0026#39;lbfgs\u0026#39;) , the groups and re_formula specify the random effects components of the model, which we do not discuss here 3.\nWe can then retrieve the fitted data:\nvcomp = pd.Series(res.vcomp, index=list(model.exog_vc.names)) var_repeat = max(float(res.scale), 0.0) var_part = max(float(vcomp.get(\u0026#34;part\u0026#34;, 0.0)), 0.0) var_appraisal = max(float(vcomp.get(\u0026#34;appraisal\u0026#34;, 0.0)), 0.0) var_interaction = max(float(vcomp.get(\u0026#34;appraisal_part\u0026#34;, 0.0)), 0.0) var_reprod = var_appraisal + var_interaction var_grr = var_repeat + var_reprod var_total = var_grr + var_part Adding uncertainty - bootstrapping #Another framework, an even more natural one would be a Bayesian one. Namely, the Bayesian framework gracefully implements multilevel models, and allows high flexibility. A known advantage of Bayesian methods is its intrisic probabilistic notions. Within this framework, it would be natural and straightforward to include uncertainties.\nWe however did not use the native probabilistic approach. In our case, we would need other methods, such as bootstrapping. Bootstrapping is a straightforward method, that consists in repeating our measurement/experiment by synthetically generating data by resampling it.\nIn our case, the idea would be to perform a sampling from the measurements for every appraisal $\\times$ part combinations.\nThe approximate code below (python pseudocode) demonstrates this idea.\n# d - the dataframe of the experiment cell_groups = list(d.groupby([\u0026#39;appraisal\u0026#39;, \u0026#39;part\u0026#39;])) for boot_id in range(n_bootstrap): resampled_cells = [] for cell_key, cell_df in cell_groups: n_rows = len(cell_df) # number of tests done in the appraisal x part combination row_ids = rng.integers(low=0,high=n_rows,size=n_rows) # generate sampling ids (with replacement) resampled_cell = cell_df.iloc[row_ids] # resample resampled_cells.append(resampled_cell) # store d_boot = pd.concat(resampled_cells) # create a new dataset model = get_model(d=d_boot) # get the statsmodels model res = model.fit( reml=True, method=\u0026#39;lbfgs\u0026#39;, start_params=res_0.params ) # fit on the new data grr_table = grr_table_from_fit(model=model,res=res,tol=tolerance) boot_records.append(grr_table) Once the bootstrapped data is obtained, one can obtain visualize the histogram. This shows how \u0026ldquo;uncertain\u0026rdquo; our estimated parameters when resampling our data.\nFigure: Histogram of bootstrapped parameters illustrating uncertainty. A working example - marimo notebook # Conclusion #In this short reading, we have defined the Gauge R\u0026amp;R method with its two main computing methods - the \u0026ldquo;classical\u0026rdquo; ANOVA and a the more \u0026ldquo;modern\u0026rdquo; Multilevel Model. We have stated the main ideas and mathematical considerations. We focused on main benefits and drawbacks of the Multilevel Model vs ANOVA and provided the squelette for its Python implementation.\nA factor in the context of DoE is the influence/regressor that we are experimenting for (e.g. when analyzing the effects of watering and light on the plant growth, water and light are the factors).\u0026#160;\u0026#x21a9;\u0026#xfe0e;\nA linear model can be defined without including the overall mean $\\mu$. In that case, this would mean that the \u0026ldquo;overall\u0026rdquo; effect is included in all of the effects\u0026rsquo; contributions.\u0026#160;\u0026#x21a9;\u0026#xfe0e;\nThe Python syntax for building statistical models is based solely on statsmodels and is very different from R. The latter is build to handle similar known cases nicely. Thus, if coming from R, the semantics can see very unnatural.\u0026#160;\u0026#x21a9;\u0026#xfe0e;\n","date":"April 4, 2026","permalink":"https://blog.leokrglv.net/posts/gauge_rr/","section":"Posts","summary":"\u003ch2 id=\"the-problem\" class=\"relative group\"\u003eThe Problem \u003cspan class=\"absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100\"\u003e\u003ca class=\"group-hover:text-primary-300 dark:group-hover:text-neutral-700\" style=\"text-decoration-line: none !important;\" href=\"#the-problem\" aria-label=\"Anchor\"\u003e#\u003c/a\u003e\u003c/span\u003e\u003c/h2\u003e\u003cp\u003eConsider the following known industrial problem - we manufacture samples (possibly with different dimensions, characteristics, etc\u0026hellip;)\nwhich we are then testing/verifying. What we want then is to characterize the measurement processes, as it may have multiple sources\nof variability. Indeed, the measurement process can depend on the operator and appraisal (i.e. testing machine), the part itself and potentially some other things that we include in our observations.\nAdditionally to the mentioned single factors, interactions could also be significant\n(e.g. some tester or operator is better off with some particular type of the part).\u003c/p\u003e","title":"Gauge R\u0026R - ANOVA \u0026 Multilevel models in python"},{"content":"I am Leo, and this is where I write about mathematics, statistics, and practical Python work.\nThe goal of the blog is not to present polished textbook chapters. It is closer to a working notebook with explanations: enough mathematical detail to know what is going on, enough code and plots to test the idea, and enough honesty about assumptions to keep the result useful.\nIn addition to plug-and-play results and illustrations, I may add learning topics throughout my journey.\nYou can start with the posts, browse the projects page for the more code-oriented pieces, or skim the notes section for short tips and tricks.\n","date":"March 14, 2026","permalink":"https://blog.leokrglv.net/about/","section":"Home","summary":"\u003cp\u003eI am Leo, and this is where I write about mathematics, statistics, and practical Python work.\u003c/p\u003e\n\u003cp\u003eThe goal of the blog is not to present polished textbook chapters. It is closer to a working notebook with explanations: enough mathematical detail to know what is going on, enough code and plots to test the idea, and enough honesty about assumptions to keep the result useful.\u003c/p\u003e\n\u003cp\u003eIn addition to plug-and-play results and illustrations, I may add learning topics throughout my journey.\u003c/p\u003e","title":"About"},{"content":"What is a group? #A group is a very abstract mathematical concept that I like to thing of as an extension to the notion of a set. That is, a set is quite an abstract mathematical concept. A set is an object that \u0026ldquo;groups\u0026rdquo; any elements by its properties, or even using any \u0026ldquo;logic\u0026rdquo; we can think of.\nOne can define a set to be, for example, all people of age 32. Mathematically, one can define a set of all real numbers, a set of all even numbers, a set of even functions, etc\u0026hellip; A set can be finite, countably infinite, uncountably infinite, etc\u0026hellip; A set can be caracterized by different things, for example the cardinality of the set. The cardinality can be thought of as the size of the set.\nNow, what is a group then? What the notion of the group does, is it establishes a relationship between the elements in the set. Namely, the set simply \u0026ldquo;contains\u0026rdquo; elements in an abstract manner. The group, however, establishes some \u0026ldquo;action/connection\u0026rdquo; between the contained elements inside this group.\nThat is, one can for example define the addition relationship (e.g. 2 real numbers can be added), multiplication relationship (e.g. 2 matrices can be multiplied), composition relationship (e.g. 2 functions can be composed) etc\u0026hellip; That is, the relationship we\u0026rsquo;re talking about is nothing but a function that takes two elements of the set and spits another. One can remember that the notion of groups is often backed up by examples from symmetries, which are nothing but functions.\nAn important property of a group is that all elements in this set can be obtained via each other. Mathematically speaking, one calls this property \u0026ldquo;closure\u0026rdquo;.\nWhat it means is that for any 2 elements of the group, their product/operation will result in another element of the group. Alternatively, any element of the group can be obtained via a binary operation between two elements of the group.\nA group is thus an abstract mathematical concept that is based on 2 notions: a set and a binary function. The function in the context of a group is called an operation. This operation maps two elements of a set to another element of this set. Therefore, a group is a set, with a defined operation (the operation can, e.g. be addition, product, composition,\u0026hellip;) that we sometimes note as $(G, \\circ)$, where $G$ is the set and the $\\circ$ corresponds to the binary operation. The formal mathematical definition of a group is therefore:\nThe group $(G, \\cdot)$ is said to be a group if it obeys 3 conditions:\n$\\exists!\\;e \\in G: \\forall g \\in G, g\\circ e = g = e \\circ g \\quad$. In other words, there exists a neutral element, such that composed with any element in the group, it will give this element (example for the group $(\\mathbb{R^*}, \\times)$, the neutral element is 1). $\\forall g\\in G, \\;\\exists! g^{-1} : g\\circ g^{-1} = g^{-1}\\circ g = e$. That is, for any element of the group, one can find its unique inverse that by operating, will yield the neutral element. $\\forall g ,h \\in G,\\; g\\circ h \\in G \\text{ and } h \\circ g \\in G$. In other words, the result of the operation between any 2 elements in $G$ will give an element that also belongs to $G$. Or more informally, we\u0026rsquo;ll repeat ourselves - it\u0026rsquo;s a set (either finite or infinite), that is closed (remember the notion of closure) under an operation, containing an identity element\n(it can be shown that can only exist one identity element) and an inverse for every element.\nThis is quite an abstract concept, but also very common, when brought it with correspondance with real examples. Examples of group in physics involve symmetry transformations (geometrical operation of a molecule that leave them invariant), Lorentz transforms, Galilean transforms, and many more.\nApart from the exact notion of the group, there are multiple concepts related to it. For example, the notion of a conjugacy class of a group.\nIntuitively, this can be thought of as a kind of a subset of this group containing elements, that share similar properties. More precisely, mathematically, one defines the conjugacy class $C$ as follows:\nTwo elements $x$ and $y$ belongs to the conjugacy class $C$: $$ y, x\\in C \\iff \\exists u \\in G :\\; u^{-1}\\circ x\\circ u=y$$ In other words, elements $x,y$ of the group $G$, belong to the same conjugacy class if there is some kind of other element $u$ of $G$, such that they are related by $u^{-1}\\circ x \\circ u = y$.\nIt is not very easy to try to understand the meaning of this notion, since the general concept of groups is quite abstract. We could interpret this as follows; let the group $(G, \\circ)$ represent some kind of action. Then, two actions $x$ and $y$ belong to the same class if performing the action $x$ results to the same action as performing some kind of action $u$, then $y$, then remove the action of the $u$ operation. In other words, the actions $x$ and $y$ are somewhat the same, in a different basis.\nWe could try to make an analogy with matrices i.e. linear transformations. That is, we know there exists such a concept as change of basis. This concept gives the possibility, to express a matrix $A$ in two different basis. This is done via the change of base matrix $P$, namely, $A' = P^{-1}AP$. The matrix $A'$ represents the exact same matrix as $A$, but in a different basis/from a different point of view.\nSymmetries #We\u0026rsquo;ve mentioned that the notion of groups is an abstract concept and can be associated to many, even non-purely mathematical conepts. For mathematics-related group examples, one can mention the $(\\mathcal{Z}, +)$ group (group of integers with the binary operation of addition), the permutation group (the group consisted of permutation operations). Here, we\u0026rsquo;ll be interested in symmetries, and describe these symmetries using the notion of groups. Symmetries are geometrical transformations that can form a group.\nExample #Let\u0026rsquo;s provide an example of such transformation. Consider a triangle with equal sides with edges denoted $A,B,C$. What are the transformations that could potentially make a group of symmetries? More simply put - what is the set of transformations that will leave the system invariant and make a group?\nFirst, we have the identity element. The identity operation does not change anything. The second operation is the rotation by $120$ degrees. This operation will map $A \\mapsto B$, $B \\mapsto C$ and $C\\mapsto A$. However, the rotation by $120$ degrees will not simply permute the edges of the triangle - it will also rotate the coordinates, that are attached to the individual atoms. Indeed, one can attach a fixed coordinate frame to every edge and make sure that these coordinates do indeed rotate with respect to the \u0026ldquo;fixed coordinate frame\u0026rdquo;. The third operation is the rotation of the triangle by -120 degrees. This would be the exact same thing as the rotation by 240 degrees. The fourth, fifth and the sixth operations are mirror operations. That is, if one reflects the triangle with respect to the bissectrice going from every single edge ($A, B$ or $C$). By defining these 6 operations, one can create a group of transformations. It is indeed a group, since for all elements, there exist an inverse element and an identity element. This group is commonly reffered to as a $C_{3v}$ group.\nFor different shapes and symmetries and even dimensions (one can indeed consider a triangle in 2D or in 3D), one can come up with different groups.\nIn order to make the notation more consise, one can make arrange them in tables, often reffered to as the Cayleigh table. The Cayleigh table keeps track of all possible operations in a group. We will create a Cayleigh for this $C_{3v}$ group later. For now, we will restrict ourselves to a more simple group.\nFor an example, one can consider a simpler group $\\mathcal{Z}_2$ commonly named as the inversion group or the permutation group of order 2, or $(\\{0,1\\},+)$ group. This is nothing but the group of addition modulo $2$. The operations between the two elements of the group can be summarized as $$ 0+0 \\stackrel{\\text{mod }2}{=} 0\\\\\\\\ 0+1 \\stackrel{\\text{mod }2}{=} 1\\\\\\\\ 1+0 \\stackrel{\\text{mod }2}{=} 1\\\\\\\\ 1+1 \\stackrel{\\text{mod }2}{=} 0 $$ One may easily identify the neutral element $0$ and note that this group is Abelian, meaning that all the elements commute. And the corresponding Cayleigh table for the group. $$ \\def\\arraystretch{1.5} \\begin{array}{|c|c|c|} \\hline \\circ \u0026 0 \u0026 1 \\\\\\\\ \\hline 0 \u0026 0 \u0026 1 \\\\\\\\ \\hline 1 \u0026 1 \u0026 0 \\\\\\\\ \\hline \\end{array} $$There are 2 conjugacy classes in this group, namely, the neutral element $\\{0\\}$ conjugacy class and $\\{1\\}$ conjugacy class. One can create the same Cayleigh table for the mentioned $C_{3v}$ group. These tables, are however, not very easy to manipulate for large groups. Indeed, for a group of $6$ elements, the table has $6\\times 6 = 36$ entries in it. There are ways, however, to represent these groups in a more compact way. We will further see the notion of the character table, which can summarize its properties in a more compact manner.\nLet\u0026rsquo;s try to write down the Cayleigh table of the $\\text{C}_{3v}$ group and analyze it using the properties we\u0026rsquo;ve defined above. So we\u0026rsquo;ve identified all the 6 transformatins belonging to this group. It can be shown that there exists only 2 types of such groups. That means that one can come up with only two \u0026ldquo;different Cayleigh tables\u0026rdquo; for a group of order 6. Up to notation of course. In order to create the table, one can write down all the transformations we\u0026rsquo;ve encountered before in a table. During the filling process, one can ask ourselves at each intersection of the table the following question:\nIf I first do [transformation from top] and then do [transformation from right], what would it be equivalent to?\nFor example, let\u0026rsquo;s take one trivial case: first apply $e$ - the identity operation, and the apply $\\sigma_A$ (the mirror with respect to the bissectrice from the point $A$)? It is clear the result will be $\\sigma_A$, since the $e$ operation doesn\u0026rsquo;t do anything\u0026hellip; What about a more complex case? First apply $C_1$-the rotation by $120$ degrees, and then the $\\sigma_A$ mirror operation? This may not be very obvious, but it is quite easy to verify by drawing the 2 transformations. In fact, the 2 transformation will create some kind of permutation, and the final permutation has the mapping: $\\sigma_A \\circ C_2: A \\mapsto C; B \\mapsto B; C \\mapsto A$. which is nothing but the $\\sigma_B$ transformation. Thus, at the intersection of $C_1$ (top - first) and $\\sigma_A$ (left - second), the resulting element will be $\\sigma_B$. By repeating this procedure $6\\times 6$ times, one can obtain the Cayleigh table for the $C_{3v}$ group:\n$\\circ$ $e$ $C_1$ $C_2$ $\\sigma_A$ $\\sigma_B$ $\\sigma_C$ $e$ $e$ $C_1$ $C_2$ $\\sigma_A$ $\\sigma_B$ $\\sigma_C$ $C_1$ $C_1$ $C_2$ $e$ $\\sigma_B$ $\\sigma_C$ $\\sigma_A$ $C_2$ $C_2$ $e$ $C_1$ $\\sigma_C$ $\\sigma_A$ $\\sigma_B$ $\\sigma_A$ $\\sigma_A$ $\\sigma_C$ $\\sigma_B$ $e$ $C_2$ $C_1$ $\\sigma_B$ $\\sigma_B$ $\\sigma_A$ $\\sigma_C$ $C_1$ $e$ $C_2$ $\\sigma_C$ $\\sigma_C$ $\\sigma_B$ $\\sigma_A$ $C_2$ $C_1$ $e$ So what can we say intuitively about this group? First, this group is not abelian, since performing 2 different operations in different order may not yield the same resulting state. Second thing is that there are 3 (2) types of \u0026ldquo;intuitive\u0026rdquo; transformations, namely, the identity transformation, the rotations (2 rotations) and reflexions (3 reflexions). One may notice that on the table, there is some kind of pattern. Namely, the table can be divided into 4 quadrants - top left (TL), TR, bottom right (BR) and BL. These quandrants can be characterized by the fact they only contain a specific type of transformations, i.e. either the $\\sigma$ \u0026rsquo;s or $C$ \u0026rsquo;s (without counting $e$). This can be intuitively interpreted as conjugacy classes, however, this is only a visual interpretation and is not always true and useful for larger groups.\nIn the $C_{3v}$ symmetry group, there are 3 conjugacy classes - the trivial one $\\{e\\}$, the rotations class $C \\equiv \\{C_1, C_2\\}$ and the reflexions class $\\sigma \\equiv \\{\\sigma_A, \\sigma_B, \\sigma_C\\}$ (Note: there is a special notation for groups and their elements, which, unfortunately, I do not fully follow). What this means is that if we take e.g. one element from the rotation class - some $C_i \\in C$, then no matter which transformation of group $u \\in C_{3v}$ we\u0026rsquo;re taking; it can be the trivial identity ($u = e$), the element from the same conjugacy class ($u = C_k \\in C$) or from the different conjugacy class ($u = \\sigma_j \\in \\sigma$), we will always get that the result of $u^{-1}\\circ C_i \\circ u$ will be in the same conjugacy class as $C_i$, namely, in the conjugacy class $C \\equiv \\{C_1, C_2, ...\\}$: $\\\\; \\\\; \\\\; u^{-1}\\circ C_i \\circ u \\in C$. Note: finding conjugacy classes is not always straightforward. They may obey to the notion of \u0026ldquo;different types of transformations\u0026rdquo; as in the example with $C_{3v}$ (identity, rotations, reflexions), but this is not always a matter of intuition. There exist databases and tables with groups and their corresponding partition into conjugacy classes.\nRepresentations #Once again, the notion of groups is very broad, and can be associated to many concepts. We\u0026rsquo;ve mentioned examples of groups, like $(\\mathcal{Z}_2, +)$, particular matrix group like the $\\text{SO}3$ group ( $3\\times 3$ matrix of determinant $1$) and many others.\nHere, the considered groups will be symmetries. That is, groups of geometrical transformations.\nRepresentation theory is in a way a branch of group theory. In representation theory, what we do is we associate groups to matrices.That is, symmetry operations are associated to groups, which then are associated to matrices.\nThus, the idea of representation theory to associate every element of a group to a matrix of any size. In addition to that, we want this association to be a homomorphism.\n$$ (G, \\circ) \\equiv \\{e, g_1, g_2, ..., g_N\\} \\xrightarrow{\\text{Mapping (homomorphism)}} \\{M_e, M_{g_1}, M_{g_2}, ...\\} $$Before going into the notion of group representation, one would need a couple of notions.\nLet $f: G \\rightarrow H$, a function that maps an element from the group $(G, \\circ)$ to the group $(H,\\star)$, i.e. $f: g \\mapsto h$, with $g\\in G$, $h\\in H$. Then the function $f$ is a homomorphism if $\\forall g_1, g_2 \\in G$, $f(g_1\\circ g_2) = f(g_1)\\star f(g_2)$.\nA homomorphism $f'$, which is one-to-one (bijective) is an isomorphism. Thus, intuitively speaking, a homomorphism is a map from a group to another group such that the operations inside the \u0026ldquo;target\u0026rdquo; group is the same as in the \u0026ldquo;original\u0026rdquo; one.\nNow, one defines the central concept, i.e. the representation of a group.\nLet $G$ some group. Let $V$ - some kind of vector space over some field of dimension $n$. A representation of the group $G$ is a function $\\Gamma$, sometimes denoted $\\Gamma(G)$, defined by $\\Gamma(G): G \\rightarrow \\text{GL}(V)$, that is, $\\Gamma(g): g \\mapsto \\text{gl}$, for some $g\\in G$ and $\\text{gl}\\in \\text{GL}(V)$. The representation $\\Gamma$ is a homomorphism.\nLet\u0026rsquo;s break down the definition. The $\\text{GL}(V)$ is the general linear group over the space V, that is, it is a set of linear transformation in the space $V$ of dimension $n$. Similarly, $\\text{GL}(V)$ in nothing but the space of $n\\times n$ invertible matrices, that act on the vector space $V$. Note that this vector space can be arbitrary, e.g. $\\mathbb{R}^n$, Hilbert space, or something else.\nIn other words, the representation is a homomorphism (a function) that will map every element $g\\in G$ to a matrix $M_g\\in \\text{GL}(V)$. The space, to which $\\Gamma$ maps the elements of $G$ is also a group (the group matrix). The values of these functions must obey the same rules as the elements $g \\in G$. Thus, when saying \u0026ldquo;representation\u0026rdquo;, one can refer to the most general $\\Gamma$ as the set of mappings from $g\\in G$ to elements in $\\text{GL}(V)$.\nLet\u0026rsquo;s once again emphasize on this: the term representation itself encompasses the mappings from elements $g\\in G$ to elements in $\\text{GL}(V)$, and NOT one particular matrix $M_g \\in \\text{GL}(V)$ associated to the element $g\\in G$.\nOne can consider an example of a simple parity group of order 2, that we already discussed. This group (sometimes denoted as $\\mathbb{Z}_2$) has 2 elements: $\\{e, p\\}$ and they follow the same multiplication rules as $\\{1, -1\\} \\equiv \\{e, p\\}$ or same additions rules as $\\{0,1\\}$ $\\text{mod} 2$. One can try to create the representation $\\Gamma$ for this group. Since the group is small, one can easily come up with a set of $2$ matrices, that will obey to the same multiplication rule as the group elements (see the Cayleigh table above). The $2$ possible matrices are given by:\n$$ \\begin{pmatrix} 1 \u0026 0 \\\\\\\\ 0 \u0026 1 \\end{pmatrix} \\equiv e, \\quad \\begin{pmatrix} 0 \u0026 1 \\\\\\\\ 1 \u0026 0 \\end{pmatrix} \\equiv p $$which indeed follow the same multiplication rules as the group $\\mathbb{Z}_2 \\equiv \\{e, p\\}$. One must note that the group itself, containing $\\{e,p\\}$ is an abstract group, which simply has multiplication rules $e\\cdot p = p\\cdot e= p$; $p\\cdot p = e$ and $e \\cdot e = e$. The group has nothing to do with matrices or even $\\mathbb{Z}_2$. Indeed, the group is anything that will obey these kind of relations. The representations, however, do indeed have a relation with matrices and $\\mathbb{Z}_2$. In other words, one can come up with 2 representations:\n$$ \\Gamma _{\\mathbb{Z} _z}: e \\mapsto 1, p\\mapsto -1 $$$$ \\Gamma _{\\text{GL}(V)}: e \\mapsto \\begin{pmatrix} 1 \u0026 0 \\\\\\\\ 0 \u0026 1 \\end{pmatrix} , p \\mapsto \\begin{pmatrix} 0 \u0026 1\\\\\\\\ 1 \u0026 0 \\end{pmatrix} $$ both representations are valid. Indeed, the two elements of $\\mathbb{Z}_2$ are indeed elements of $\\text{GL}(V)$ of dimension 1 and the two matrices are elements of $\\text{GL}(V)$ of dimension 2. We will then see that it is possible to come up with another one.\nIn quantum mechanics, as said, we\u0026rsquo;re working with finite groups, that are groups of symmetry. This group of $2$ elements represents the symmetry of inversion.\nNow, one should introduce the very important concept of reductibility. Consider a symmetry group $G$ that describes some system. And let $\\Gamma$ be its representation. Let now $\\Gamma (g)$ be a representaion, that acts on the vector space $V$. One says that this representation is completely reducible if there exists a non-zero subspace $W \\in V$, that is invariant under $\\Gamma$. In other, words, it is reducible if there exists a subspace $W \\in V$, such that $\\forall \\ket{w} \\in W$, such that $\\Gamma (g) \\ket{w} \\in W$, which is true FOR ALL $g\\in G$. In addition to that, the subspace $W^\\text{T}$ is orthogonal to $W$. One can visualize it in terms of matrices. That is, $\\Gamma (g)$ is reducible if for all $g\\in G$, one can put the matrices $\\Gamma$ in the form $$ \\Gamma (g) \\equiv \\begin{pmatrix} \\Gamma(g) ^{(1)} \u0026 0 \\\\\\\\ 0 \u0026 \\Gamma(g) ^{(2)} \\\\\\\\ \\end{pmatrix} = \\Gamma(g) ^{(1)} \\oplus \\Gamma(g) ^{(2)} $$ Where the two matrices $\\Gamma (g) ^{(1),(2)}$ are also matrices and the matrix $\\Gamma (g)$ is diagonal by blocks. A reducible representation is a representation that can be block-diagonalized more and more. A representation that is already \u0026ldquo;maximally reduced\u0026rdquo;, is called irreducible representation.\nSince the representation we\u0026rsquo;re considering are matrices, there should be a way to go from a reducible to irreducible representation. This could be done via a change of basis matrix. Namely, one can write $$ \\Gamma(g) _{\\text{red}} = S^{-1} \\Gamma(g) _{\\text{irred}} S $$This relation is called an equivalence relation. That is 2 representations are equivalent if there exists a basis that transforms them as given above. This relation is called a similarity transformation. In other words, 2 equivalent representations are related via a similarity transformation.\nIn representation theory, we\u0026rsquo;re interested in non-equivalent, but irreducible representations. That is, in representation theory, two representations, are considered to be \u0026ldquo;the same\u0026rdquo; if they are irreducible and possibly equivalent. One should, however, note another fact concerning the reducible representations. The general matrix form of the irreducible representation is instead given by $$ \\begin{pmatrix} \\Gamma(g) _1 ^{(1)} \u0026 0 \u00260 \u00260 \u0026 0 \u0026 0\\\\\\\\ 0 \u0026 \\Gamma(g) _2 ^{(1)} \u00260 \u00260 \u00260 \u0026 0\\\\\\\\ 0 \u0026 0 \u0026 \\ddots \u0026 0 \u00260 \u00260 \\\\\\\\ 0 \u0026 0 \u0026 0 \u0026 \\Gamma(g) _1 ^{(2)} \u00260 \u00260 \\\\\\\\ 0 \u0026 0 \u0026 0 \u0026 0\u0026 \\Gamma(g) _2 ^{(2)} \u0026 0 \\\\\\\\ 0 \u0026 0 \u0026 0 \u0026 0 \u0026 0 \u0026 \\ddots \\end{pmatrix} $$That is, the representation, in its most general case, is decomposed as\n$$ \\Gamma(g) = \\bigoplus_{a,x} \\Gamma(g)^{(a)}_{x} $$instead of simply $\\Gamma(g) = \\bigoplus _{a} \\Gamma(g)^{(a)}$. That is, one adds some degenerescence to a given element of the block-representation $\\Gamma(g)^{(a)}$. Meaning that some of the irreducible representations are encountered more than once in the final, irreducible representation.\nThis is a very important formula that describes the notion of (ir)-reducible representations.\nWhy do we need it? Recap up to now # So, for now, what we understand is that there exists some kind of spatial structure (a triangle, cube, molecule, etc\u0026hellip;) that has some symmetry associated to it. What symmetry means is that one can apply one of the symmetrical transformation, and we won\u0026rsquo;t be able to tell the difference between the systems before and after transformations. This is called \u0026ldquo;invariance under transformations\u0026rdquo; or simply \u0026ldquo;symmetries\u0026rdquo;.\nThese symmetries obey to some mathematical properties between themselves. These mathematical properties are called a group. In other words, the geometrical transformations may obey relations, that are nothing but groups.\nInstead of working with the group, which can be characterized using a Cayleigh table (containing all possible relations in the group), one can characterize the group using the representation theory. This is equivalent, since a representation is an isomorphism by definition. That means that the relations of all the elements with all the elements is the same as the relations between the matrices that form this representation (recall that a representation is mainly a association between abstract elements of a group to \u0026ldquo;real\u0026rdquo;/\u0026ldquo;tangible\u0026rdquo; matrices).\nThe set of matrices (the representation) can be different. Indeed, the matrices is a representation as long as it obeys the necessary relation from the group.\nBut what could we eventually need this for? Suppose the (symmetric) system that we are analyzing is somewhat a physical object (most likely a molecule). We want to retrieve and compute some of its properties. This will be most likely done through a Hamiltonian operator, which can be represented as a matrix. The Hamiltonian will also most likely lead to the solution of an eigenvalue problem, thus requiring diagonalization. The next question is thus \u0026ldquo;what does the Hamiltonian have to do with the symmetry transformation matrix (representation)?\u0026rdquo; Well, the answer to this makes sense - they commute. Indeed, the symmetry transformation is by definition a transformation that takes into account symmetry and the system is unvariant under this transformation, meaning the Hamiltonian does not change either. From linear algebra, this commutative property leads to another mathematical property: if 2 operators are commuting, it is possible to find a common basis, such that both of the commuting operators will by (block) diagonal. So, if the symmetry matrix is nice and easy, one can diagonalize the latter and work our way to the diagonalization of the hamiltonian itself.\nNow, what does the notion of reducible/irreducible has to do with that? When discussing the representations, we mentioned that we\u0026rsquo;re especially interested in irreducible representations, which, by definition are block-diagonal. This means that irreducible representations are matrices of symmetry transformations (i.e. obeying the group rules), but meant to be easy manipulated.\nThus the goal of the representation theory is:\nbased on symmetry considerations, find a valid representation, and then reduce it in order to obtain the simplest form possible using maths from representation theory. Once this is done, it is possible to infer some properties of the system, and potentially solve the eigenvalue problem for the complex hamiltonian, since the irreducible representation have a simpler structure.\nMathematical description #Before going into the practical examples, trying to understand the role of the notion of group representations and how could that be useful, one should simply mention, without proving, some of important theorems \u0026amp; concepts. These concepts are tools, that are used to accomplish the goal that we\u0026rsquo;ve written above.\nShur Lemma\u0026rsquo;s #The Shur\u0026rsquo;s Lemma\u0026rsquo;s is the central theorem in the group representation theory. Namely, many results can be simply derived from the definitions and the mentioned Shur\u0026rsquo;s lemmas. For Shur\u0026rsquo;s Lemmas, let $\\Gamma_1$ and $\\Gamma_2$ - two different irreducible representations each belonging to its own vector space $V_1$ and $V_2$ respectively\n(for example, the two representations could be matrices of size $3\\times 3$ and $2\\times 2$ acting on $\\mathbb{C}^3$ and $\\mathbb{C}^2$). Let in addition $M$ be a operator (matrix) $M: V_1 \\mapsto V_2$, such that it commutes with both of the representations $\\Gamma_{1,2}$. That is, $M\\;\\Gamma_1(g) = \\Gamma_2(g)\\; M \\;\\; \\forall g \\in G$, then\nShur\u0026rsquo;s Lemma 1 Shur\u0026rsquo;s Lemma 2 If $\\Gamma_1$ and $\\Gamma_2$ are not-equivalent, then $M=0$ If $\\Gamma_1 = \\Gamma_2 \\equiv \\Gamma$, then $M$ can only be the multiple of identity, that is, $M$ has the form $M\\equiv \\lambda I$ These two lemmas state that if some operator commutes with some two representations of a group, then this operator is either zero (the two representations are equivalent) or diagonal (if they are the same).\nNote: We mentioned in one of the previous sections how representations are useful. Here, the matrix $M$ in question can be nothing but our Hamiltonian.\nCharacters #By now, we came across some mathematical objects such as group, subgroup, isomorphism, representation. It is now time to add a new one - the character. We\u0026rsquo;ve mentioned that in representation theory, we\u0026rsquo;re interested in representations, up to an equivalence relations (i.e. equivalent representations are considered to be the same). Thus, we may want to somehow characterize a representation (remember the definition of the representation - set of matrices), without having to deal the problem of cheking the equivalence. The answer to that is the trace of the matrix. The set of traces of a representation is its character. One denotes a character of a matrix $\\chi_\\Gamma(g)$ of the representation $\\Gamma$ of the matrix $\\Gamma(g)$. That is, $\\chi_\\Gamma(g)=\\text{Tr}[\\Gamma(g)]$. One should recall the properties of the trace - $\\text{Tr}(A\\oplus B) = \\text{Tr}(A)+\\text{Tr}(B)$ and $\\text{Tr}(A\\otimes B) = \\text{Tr}(A)\\cdot \\text{Tr}(B)$.\nOrthogonality #We mentioned that should be able to decompose a representation into a direct sum of irreducible representations. That is, we want to decompose the representation (which possibly can be reduced) into the direct sum of irreducible: $\\Gamma = \\bigoplus_{a,x}\\Gamma_{a,x}$, with $a$ - the irreducible representations and $x$ - the corresponding multiplicities or $\\Gamma = \\bigoplus_{a} b_a \\Gamma_{a}$ and $b_a$ - the multiplicities. The resulting representation consists of a direct sum of vector spaces $\\bigoplus_{a,x} V_{a,x}$. For every space $V_{a,x}$, one my identify a basis $\\{\\ket{j}\\}$. One can denote the basis $\\{\\ket{j}\\}$ belonging to the space $V_{a,x}$ as $\\{\\ket{a,j,x}\\}$. Thus the basis of a given representation is denoted as $\\{\\ket{a,j,x}\\}$.\nThe orthogonality theorem is an important result, that can be shown using the Shur\u0026rsquo;s lemmas (not done here)\nLet $\\Gamma_a$ and $\\Gamma_b$ - two non-equivalent representations over the space $V_a$, $V_b$ of dimensions $n_a$ and $n_b$, of the group $G$ of order $N$. Then, the great orthogonality theorem states:\n$$ \\sum_{g \\in G}\\frac{n_a}{N} [\\Gamma_a(g)]_{i,k}^{*} [\\Gamma_b (g)] _{m,n} = \\delta _{a,b} \\delta _{i,m} \\delta _{k,n} $$Where the sum is performed over the group elements $g\\in G$. This can be easily rewritten as\n$$ \\sum_{g \\in G} \\sqrt{\\frac{n_a}{N}} [\\Gamma_a (g)]_{i,k}^{*} \\sqrt{\\frac{n_b}{N}} [\\Gamma_b (g)] _{m,n} = \\delta _{a,b} \\delta _{i,m} \\delta _{k,n} $$and identifying the mentioned elements of the vector space\n$$ \\ket{a,k,j} \\equiv \\sqrt{\\frac{n_a}{N}} [\\Gamma_a (g)]_{j,k} $$The basis ortogonality relation can be identified as $\\braket{b,i,k|a,m,n}=\\delta_{a,b}\\delta_{i,m}\\delta_{k,n}$.\nBased on the orthogonality relations, it is possible to derive the Burnside lemma, which gives a restriction to dimensions of the irreducible vector spaces $n_a$: $$ \\sum_{a}n_a^2 = N $$ That being said, the sum of squares of the dimensions of the vector spaces into which the representations are operating is equal to the order of the group.\nThe Burnside Lemma is quite useful for determining the dimensions of representations of groups of small order. That is, suppose there exists a group of order $6$. Then, based on Burnside\u0026rsquo;s lemma, one state that a possible irreducible representation will be made of $6$ representations of dimension $1$. Indeed, $\\sum_{a\\in |\\Gamma_a|}n_a^2=6\\cdot 1^2 = 6 = N$, where $|\\cdot|$ stands for cardinality/size and $N$ the size of the group. Or, another possibility is to have $1$ representation of dimension $2$ and $2$ representations of dimension $1$. Indeed, $\\sum_{a\\in |\\Gamma_a|}n_a^2=1^2+1^2+2^2=6=N$. Similarly, for e.g. a group of size $8$, based on only Burnside, there can be either $8$ irreducible representations of dimension 1 ($8\\cdot 1^2 = N$), or $1$ representation of dim. $2$ and $4$ of dim. $1$ ($2^2 + 1^2+ 1^2+ 1^2+ 1^2 = N$) or simply $2$ representations of dimension $2$.\nIf in addition, we know, how many conjugacy classes there is, it is even easier to determine the dimensions of the representations. That is, suppose some kind of group of order $10$ contains $4$ conjugacy classes. Then, there are $4$ irreducible representations. It is straightforward to find out the dimensions of the $4$ irreducible representations - they are $2^2+2^2+1^2+1^2 = N = 10$.\nIf one takes the trace of the mentioned orthogonality relation, one gets the second orthogonality theorem relation: $$ \\sum_{g\\in G} \\chi_a(g) \\chi_b(g) = N\\delta_{a,b} $$ or equivalently, one can replace the group elements $g\\in G$ by the elements of the conjugacy class, since we remember that the characters are same for all the elements in the conjugacy class. Thus, the equivalent relation is given by\n$$ \\sum_{\\mu \\in N_{\\text{conj. cl.}}} n_{\\mu} \\chi_{a}^*(C_\\mu)\\chi_{b}(C_\\mu)=N\\delta_{a,b} $$ with $n_\\mu$ - the number of elements in the conjugacy class.\nThe degeneracies $b_a$ mentioned before can be computed using the relation\n$$ b_a = \\frac{1}{N}\\sum_{\\mu} n_\\mu \\chi_{a}^*(C_\\mu)\\chi_{\\Gamma}(C_\\mu) $$ with $\\chi_{\\Gamma}(C_\\mu)$ the character of the $C_\\mu$ conjugacy class.\nOne may add the last relation that gives necessary and sufficient conditions for a representation to be irreducible. $$ \\sum_{\\mu} n_\\mu |\\chi_\\Gamma(C_\\mu)|^2 = N $$Projectors #Projectors are operators that let us decompose and reduce a representation. Recall the goal of problem related to representation theory - to reduce a representation of some symmetric system. Mathematically, given a representation of a symmetry group $\\Gamma$, one wants obtain the direct sum decomposition: $$ \\Gamma = \\bigoplus_{a,x} \\Gamma_{a,x} = \\bigoplus_{a} b_a \\Gamma_a $$The space that the resulting representation will be acting on will be the exact same direct sum of $V_a$ spaces, each being orthogonal. This is equivalent of finding a representation in the basis that we called $\\{\\ket{a,j,x}\\}$. Let\u0026rsquo;s now see how we\u0026rsquo;ll try to define the projectors. We start by considering a vector $\\ket{a,j,x}$ and the representation $\\Gamma$. What happens, when we apply the representation to this element $\\Gamma (g) \\ket{a,j,x} $? Well, we remember that the $a$ in the ket notation represents the number of the non-equivalent irreducible representation, that the initial representation was reduced to. The $x$ represents its multiplicity and only the $j$ represents the \u0026ldquo;actual\u0026rdquo; vector inside this $V_{a,x}$ vector space. The usual matrix multiplication definition: $$ \\Gamma \\ket{a,j,x} = \\sum_{k=1} (\\Gamma)_{k,j} \\ket{a,k,x} $$But we know that for a given $a$ and $x$, the spaces are orthogonal, so the only non-zero contribution comes from the $\\Gamma_{a,x}$ representation:\n$$ \\Gamma \\ket{a,j,x} = \\sum_{k=1} (\\Gamma)_{k,j} \\ket{a,k,x} $$Using this and the previous orthogonality relations, one can transform it by doing some algebra into: $$ \\sum_{g \\in G} [ \\Gamma_b (g)]^{*}_{k',j'} \\; \\Gamma (g) \\ket{a,j,x} = \\frac{N}{n_a} \\delta _{a,b} \\delta _{j,j'} \\ket{a,k',x} $$from which, one defines the projection operator $\\hat{\\Pi}_{k,j}^{b}$, which is defined as\n$$ \\hat{\\Pi} ^{b} _{k,j} = \\frac{n_a}{N} \\sum _{g \\in G} [\\Gamma _b (g)]^{*} _{k,j} \\Gamma _{g} $$which will satisfy the following properties:\n$$ \\hat{\\Pi} ^{a} _{k,j} \\ket{a,j,x} = \\ket{a,k,x} $$$$ \\hat{\\Pi} ^{a} _{k,j} \\ket{a,j',x} = 0 $$This projector $\\hat{\\Pi}_{k,j}^{a}$ projects an arbitrary vector onto the vector $\\ket{a,k,x}$. If applied on another, orthogonal vector of $\\ket{a,j,x}$, the result will be zero.\nIdea of how use the representations? #Now, with all that in mind, one can complete the discussion on how are representations useful and used.\nLet\u0026rsquo;s suppose we\u0026rsquo;re given some kind of symmetric system that represents, for example, some kind of molecular structure or multiple moleculesm, that is unvariant under some kind of transformations. The first thing is to identify the abstract symmetry group. Once identified, one should find (or construct the character table (see below)). The character table gives almost all the information needed for reducing the representation. For example, what if one wants determine the degeneracies of the eigenvalue problem? That is, how many times is a certain energy degenerate. Remember, the number of degeneracy of an operator is equal to the dimension of the eigenspace. Now, we remember that the representation of the system WILL commute with the Hamiltonian. Thus, by identifying the multiplicities $b_a$\u0026rsquo;s (see identity above), one can find out the degeneracy of the subspace, associated to the same representation.\nOne often may want to solve the motion of the system (e.g. $x(t)$, $p(t)$ or any general function $\\psi(x,t)$). Then, once again, using the fact that the commutator of the Hamiltonian and $\\Gamma$ is zero, one finds the seeked solution of the general eigenvalue problem, encountered in quantum mechanics $\\hat{H}\\ket{\\psi} = E\\ket{\\psi}$. To solve this, one can consider the matrix $\\Gamma$ instead and compute the eigenvectors and eigenspaces. The latter is often performed using the projectors.\nWorked examples #We\u0026rsquo;re finally in a position of starting considering concrete examples and apply the mentioned concept for solving quantum mechanical problems. This is the most important part of the article, where we will try to understand the mentioned notions by applying them, since the considered concepts tend to be quite abstract.\nSimplest group - reflection group #We start off by considering the simplest, already mentioned group - the reflection group/ parity group/ $C_s$ group. This group has lots of names, since, as we\u0026rsquo;ve seen a group can be associated to many different notions. The main point is that the $C_s$ group has $2$ elements and there is only one possible group of order $2$. It\u0026rsquo;s Cayleigh table is given by\n$$ \\def\\arraystretch{1.5} \\begin{array}{|c|c|c|} \\hline \\circ \u0026 e \u0026 p \\\\\\\\ \\hline e \u0026 e \u0026 p \\\\\\\\ \\hline p \u0026 p \u0026 e \\\\\\\\ \\hline \\end{array} $$ This is a very simple group and it is quite easy to determine the irreducible representations. Indeed, let\u0026rsquo;s recall the burnside lemma. From the Burnside lemma, one can determine that there are $2$ irreducible representation of dimension $1$, i.e. $\\Gamma = \\Gamma_1 \\oplus \\Gamma_2$, with both $\\Gamma_1$ and $\\Gamma_2$ of dimension $1$. Thus the representation are simply $2$ numbers. It is quite easy to find that the 2 representations the following:\n$$ \\def\\arraystretch{1.5} \\begin{array}{|c|c|c|} \\hline \u0026 e \u0026 p \\\\\\\\ \\hline \\Gamma_1 \u0026 1 \u0026 1 \\\\\\\\ \\hline \\Gamma_2 \u0026 1 \u0026 -1 \\\\\\\\ \\hline \\end{array} $$ We can check that this is true - if $e \\mapsto 1$, and $p \\mapsto 1$, the relations within the group are verified. Similarly, if $e \\mapsto 1$ and $p \\mapsto -1$, the relation is still verified. Now, what about the most powerful tool available - the character table? What does it look like? Well, in order to find out, one must simply take the trace of the representaions. Here, the representations are simply numbers, thus the trace is equal to the number itself. So the character table:\n$$ \\def\\arraystretch{1.5} \\begin{array}{|c|c|c||c|} \\hline \\chi \u0026 e \u0026 p \u0026 \\text{Linear funcs. and rotations } \\\\\\\\ \\hline \\Gamma_1 \u0026 1 \u0026 1\u0026 x,y, R_z \\\\\\\\ \\hline \\Gamma_2 \u0026 1 \u0026 -1\u0026 z, R_x, R_y\\\\\\\\ \\hline \\end{array} $$ Well, this was quite easy\u0026hellip; However, let\u0026rsquo;s suppose we don\u0026rsquo;t know the irreducible decomposition, and we try to start from the physical situation. We have a simple molecule having $2$ atoms - left and right ones denoted as $L$ and $R$. Their coordinates are given by $L \\equiv (0, 0, z=-z_0)$ and $R \\equiv (0, 0, z=z_0)$. This can be a 3-dimensional system, but we can always choose a coordinate system so that only one of the components is non-zero; in this case, we take $z$.\nNow, we ask ourselves an important question - what will happen when the reflection transformation is applied? We have that the left one becomes the right one and the right one becomes the left one. Thus, the 2 transformations in the matrix form (the ID transformation and the permutation transformation) are given by $$ e \\equiv \\begin{pmatrix} 1 \u0026 0 \\\\\\\\ 0 \u0026 1 \\\\\\\\ \\end{pmatrix} \\\\;\\\\;\\\\;\\\\;\\\\;\\\\;\\\\;\\\\; p \\equiv \\begin{pmatrix} 0 \u0026 1 \\\\\\\\ 1 \u0026 0 \\\\\\\\ \\end{pmatrix} $$Now, one considers the character table that we\u0026rsquo;ve provided above and observe that there are 2 conjugacy classes: $\\lbrace e \\rbrace$ and $\\lbrace p \\rbrace$. Therefore, one can apply the different concepts we\u0026rsquo;ve mentioned before to determine the different vibrations of the molecule.\nWe know that there are 2 conjugacy classes thus we know that we will have $2$ representations of dimensions $1$. We can verify it mathematically, using the formula for $b_1$ and $b_2$. Let\u0026rsquo;s start with the formula:\n$$ b_a = \\frac{1}{N}\\sum_{\\mu} n_\\mu \\chi_{a}^*(C_\\mu)\\chi_{\\Gamma}(C_\\mu) $$then, using $N=2$ and $a \\equiv 1$ $$ b_1 = \\frac{1}{2}\\sum_{\\mu} n_\\mu \\chi_{1}^*(C_\\mu)\\chi_{\\Gamma}(C_\\mu) = \\frac{1}{2} ( 1\\cdot 2 \\cdot 1 + 1\\cdot 0 \\cdot 1 ) = \\frac{2}{2} = 1 $$ where we have used that $\\chi_{1} = 2$ (trace of matrix associated to $e$) and $\\chi_2 = 0$, and the character of $\\Gamma$ $\\chi_{\\Gamma}(C_\\mu)$ is taken from our character table. Similarly,\n$$ b_2 = \\frac{1}{2}\\sum_{\\mu} n_\\mu \\chi_{2}^*(C_\\mu)\\chi_{\\Gamma}(C_\\mu) = \\frac{1}{2}(1 \\cdot 2 \\cdot 1 + 1\\cdot 0 \\cdot (-1)) = 1 $$ which shows our initial Burnside lemma of $2$ representation of dimensions $1$.\nRemember we mentioned that we can come up with different representations. Here, we can add another one - namely, we can consider the problem in terms of coordinates transformations, rather than in terms of element permutation. The transformation will keep $(x,y)$ unchanged and the will flip the $z$ coordinate. The resulting transformation matrix is therefore given by $$ \\begin{pmatrix} 1 \u0026 0 \u0026 0\\\\\\\\ 0 \u0026 1 \u0026 0 \\\\\\\\ 0 \u0026 0 \u0026 1 \\\\\\\\ \\end{pmatrix} \\equiv e, \\quad \\begin{pmatrix} 1 \u0026 0 \u0026 0\\\\\\\\ 0 \u0026 1 \u0026 0 \\\\\\\\ 0 \u0026 0 \u0026 -1 \\\\\\\\ \\end{pmatrix} \\equiv p $$If using the formulae for $b_a$ - we will find out different things - instead of $\\Gamma = \\Gamma_1\\oplus \\Gamma_2$ we find $\\Gamma = 2\\Gamma_1\\oplus \\Gamma_2$. This is due to the fact that in the second case, we\u0026rsquo;ve implicitly converted the problem into 3 dimensions. We should also mention the meaning of the 4th column in the character table of the $C_s$ group. That is, the meaning of those \u0026ldquo;function basis\u0026rdquo; and \u0026ldquo;rotations\u0026rdquo;. One should read them as follows: the $\\Gamma_i$ transforms as \u0026lt;the function in question\u0026gt;. So, the $\\Gamma_2$ transforms as $z$, which is very obvious since the $\\Gamma_2$ flips the sign of the $z$; also, the $\\Gamma_2$ transforms as $R_x$, $R_y$, since flipping the sign would do essentially the same as rotating by $\\pi$ over the $x$ or $y$ axis. Similar reasonings can be applied for $\\Gamma_1$. So what to do, once the multiplicities have been found? Once the multiplicities have been found, we want to finally find the common eigenbasis for the obtained representations $\\Gamma = \\bigoplus_{a,x} \\Gamma_{a,x}$ and the Hamiltonian, which is our final goal. This is done through projectors. This is not very complex, but relatively computationally expensive. We will thus not discuss it there, but only the idea:\nSo the idea is to find the set of eigenvectors that will span every irreducible representation. For that, one should construct the projector operators for each representations $\\hat{\\Pi}$, and find the eigenbasis for each of them. The number of the eigenvectors, serving as the basis will be equal to the degeneracies of the computed irreducible representations, that we computed.\nThe D3v group #Now, we can proceed with the next example and the related problem. This problem comes actually from one of my previous year\u0026rsquo;s exam - on representation group theory. The statement of the problem is as follows:\nWe consider a molecule of 3 identical atoms disposed on a equilateral triangle in 3D space. The symmetry group of an equilateral triangle in 3D is the $D_{3v}$. We consider the $\\hat{z}$ direction to be perpendicular to the triangle plane. The character table of the $D_{3v}$ group is given below. The group consists of the identity operation $E$, two rotations around the $\\hat{z}$ axis $C_2$ (as in the $C_{3v}$ group). Three rotations around the 3 possible bissectrices from each vertex $C_3$. One reflection (mirror) operation along the plane, which is parallel to the plane of the triangle. Then two improper rotations that we denote $S_3$, which are given by the composition of $C_3\\circ \\sigma_h$. Finally, three reflections over the 3 plans along the bissectrices. Every atom has 3 degrees of freedom and can move in 3D. The task is to compute the characters of the representation $\\Gamma$, find the decomposition to determine the potential degeneracies.\nOkay, let\u0026rsquo;s start by providing the character table of the $D_{3v}$ group: $$ \\def\\arraystretch{1.5} \\begin{array}{|c|c|c|c|c|c|c|c|c|} \\hline \\chi_{D_{3v}} \u0026 E \u0026 2C_3 \u0026 3C_2 \u0026 \\sigma_h \u0026 2S_3 \u0026 3\\sigma_h \u0026 \u0026\\\\\\\\ \\hline \\Gamma_1 \u0026 1 \u0026 1 \u0026 1 \u0026 1 \u0026 1 \u0026 1 \u0026 \u0026 x^2+y^2, z^2\\\\\\\\ \\hline \\Gamma_2 \u0026 1 \u0026 1 \u0026 -1\u0026 1 \u0026 1 \u0026 -1\u0026 R_z \u0026 \\\\\\\\ \\hline \\Gamma_3 \u0026 2 \u0026-1 \u0026 0 \u0026 2 \u0026-1 \u0026 0\u0026 (x,y) \u0026 (x^2-y^2, xy) \\\\\\\\ \\hline \\Gamma_4 \u0026 1 \u0026 1 \u0026-1 \u0026-1 \u0026-1 \u0026 1\u0026 \u0026 \\\\\\\\ \\hline \\Gamma_5 \u0026 1 \u0026 1\u0026-1 \u0026-1 \u0026 -1\u0026 1\u0026 z \u0026 \\\\\\\\ \\hline \\Gamma_6 \u0026 2 \u0026-1\u0026 0 \u0026 -2 \u0026 1 \u0026 0\u0026 (R_x, R_y) \u0026 (xz, yz) \\\\\\\\ \\hline \\end{array} $$Okay, we may ask ourselves a question - what is the difference between the rotation around one of the bissectrice $C_3$ and the reflection operations $\\sigma_h$? Indeed, when performing the $C_2$ and $\\sigma_v$ operations, the positions of the atoms are the same, as shown on the image:\nHowever, they do differ and we can illustrate it by attaching a local coordinate system to each of the atoms.\nHere, we see the difference - the $\\sigma_v$ simply changes the red and green atoms (simple permutation operation). The $C_2$ operation, however, performs the full rotation of the local coordinate system. Try to picture the $C_2$ operation - the triangle rotates around the blue bissectrice axis, thus resulting in the flipped $z$ axis. The $\\sigma_v$, however, simply flips the $x$ axis (we keep the correct orientation of the coordinate system). We thus see that there is indeed a difference between the 2 rotations.\nOkay, so we can get back to the discussion and the problem itself. How can we determine the representation of the problem in question? This will be done via the tensor product of matrices. For that, we can even create a new subsection for that.\nTensor product #So, in order to determine the representations, i.e. $\\Gamma(E)$, $\\Gamma(C_3)$ etc\u0026hellip; we need to determine 2 matrices for each transformation - one $3\\times 3$ matrix that will describe the coordinate transformations (e.g. a rotation in 2D will yield a simple 2D rotation matrix) and the second $3\\times 3$ permutation matrix that will provide the information on how the atoms are permuted as the result of the transformation. So let the coordinate transformation matrix be $R$ and the permutation matrix $T$; then resulting representation matrix of the transformation will be given by $\\Gamma(g) = T\\otimes R$ yielding a $9\\times 9$ matrix. This does indeed make sense - there are 3 atoms, each having 3 coordinates, thus giving 9 \u0026ldquo;parameters\u0026rdquo;.\nFor that, let\u0026rsquo;s start by the simplest example - the representation of the identity transformation $\\Gamma(E)$. The permutation matrix is easily given by $$ T_E \\equiv \\begin{pmatrix} 1 \u0026 0 \u0026 0\\\\\\\\ 0 \u0026 1 \u0026 0 \\\\\\\\ 0 \u0026 0 \u0026 1 \\\\\\\\ \\end{pmatrix} $$since no atoms are permuted (changing place). Similarly, for the coordinate matrix, it is given by $$ R_E \\equiv \\begin{pmatrix} 1 \u0026 0 \u0026 0\\\\\\\\ 0 \u0026 1 \u0026 0 \\\\\\\\ 0 \u0026 0 \u0026 1 \\\\\\\\ \\end{pmatrix} $$Thus giving the representation $\\Gamma(E) = I_{9\\times 9}$. The next operation to analyze is the $C_3$ operation. We want to find the representation $\\Gamma(C_3)$. What will be its permutation matrix? So, one of the elements of the $C_3$ will perform a rotation by $120^{\\circ}$. This will make the permutation $R\\mapsto B$, $B \\mapsto G$ and $G\\mapsto R$ (where the letters represnt the colors of the atoms on the image). Therefore, the resulting permutation matrix will be given by $$ P_{C_3} \\equiv \\begin{pmatrix} 0 \u0026 0 \u0026 1\\\\\\\\ 1 \u0026 0 \u0026 0 \\\\\\\\ 0 \u0026 1 \u0026 0 \\\\\\\\ \\end{pmatrix} $$ What about the coordinate transformation matrix for the $C_3$ transformation? For that we ask ourselves - how do the transformations change the coordinates? The $z$ coordinate does not change at all, and the rest is changed as a usual 2D rotation, which is given by $$ R_{C_3} \\equiv \\begin{pmatrix} \\cos(\\theta) \u0026 -\\sin(\\theta) \u0026 0\\\\\\\\ \\sin(\\theta) \u0026 \\cos(\\theta) \u0026 0 \\\\\\\\ 0 \u0026 0 \u0026 1 \\\\\\\\ \\end{pmatrix} $$ where the angle $\\theta = 120^{\\circ}$, yielding the rotation matrix $$ R_{C_3} \\equiv \\begin{pmatrix} -\\frac{1}{2} \u0026 -\\frac{\\sqrt{3}}{2} \u0026 0\\\\\\\\ \\frac{\\sqrt{3}}{2} \u0026 -\\frac{1}{2} \u0026 0 \\\\\\\\ 0 \u0026 0 \u0026 1 \\\\\\\\ \\end{pmatrix} $$ The resulting representation matrix of the operation $C_3$ is thus the tensor product of $P_{C_3}$ and $R_{C_3}$. We can proceed further for this procedure for other operations in the list. Let\u0026rsquo;s quickly go over other example of constructing matrices. The $C_2$ rotation is the rotation over the bissectrice axis. Let\u0026rsquo;s, for example, take the blue axis. Then, the rotation will be performed around the $y$ axis, meaning that the coordinate transformation matrix will be in the form of $$ \\begin{pmatrix} \\cos(\\theta) \u00260 \u0026 \\sin(\\theta) \\\\\\\\ 0 \u0026 1 \u0026 0 \\\\\\\\ -\\sin(\\theta) \u00260 \u0026 \\cos(\\theta) \\\\\\\\ \\end{pmatrix} $$ with $\\theta = 180^{\\circ}$ yielding the matrix\n$$ \\begin{pmatrix} -1 \u00260 \u0026 0 \\\\\\\\ 0 \u0026 1 \u0026 0 \\\\\\\\ 0 \u00260 \u0026 -1 \\\\\\\\ \\end{pmatrix} $$and the permutations are given by $R\\mapsto G$, $G\\mapsto R$ and $B \\mapsto B$.\nWe must recall, that we\u0026rsquo;re mainly interested in characters, rather than the full set of all matrices (the full representation).\nWe will not write down all the permutations and coordinate transformation matrices. We\u0026rsquo;ve written down some of them - but it is quite straightforward to go on for other matrices. We will state the resulting character table: $$ \\def\\arraystretch{1.5} \\begin{array}{|c|c|c|c|c|c|c|} \\hline \\chi \u0026 E \u0026 2C_3 \u0026 3C_2 \u0026 \\sigma_h \u0026 2S_3 \u0026 3\\sigma_v \\\\\\\\ \\hline \\Gamma_{P} \u0026 3 \u0026 0 \u0026 1\u00263 \u00260 \u00261 \\\\\\\\ \\hline \\Gamma_{R} \u0026 3 \u0026 0 \u0026-1 \u00261 \u0026-2 \u00261\\\\\\\\ \\hline \\Gamma \u0026 9 \u0026 0 \u0026 -1 \u0026 3 \u0026 0 \u0026 1\\\\\\\\ \\hline \\end{array} $$ Finally, we can now use the characters of our representation, that we computed using the tensor product of permutation matrix and coordinate transformation matrix. We can thus find the desired multiplicities using the already very familiar formula $$ b_a = \\frac{1}{N} \\sum_{\\mu}^{N_c}n_{\\mu} \\chi_a^*(C_{\\mu}) \\chi(C_{\\mu}) $$ where $\\chi(C_{\\mu})$ - the characters obtained via our procedure (the first table above). The $n_\\mu$ - the number of elements inside this conjugacy class. Finally, the $\\chi_{a}(C_{\\mu})$ - the character of the irreducible representation $a$, which is found in the character table of the $D_{3v}$ group.\nConclusion #This article\u0026rsquo;s goal was mainly to make a recap of the topic of representation group theory basic applications in (quantum) physics. For that reason, we\u0026rsquo;ve tried to introduce in the most gentle and intuitive way possible. We\u0026rsquo;ve introduced the notion of representation in the same way. All the associated notions were introduced in the same way, with examples and analogies.\n","date":"May 17, 2023","permalink":"https://blog.leokrglv.net/posts/group_theory/","section":"Posts","summary":"\u003ch2 id=\"what-is-a-group\" class=\"relative group\"\u003eWhat is a group? \u003cspan class=\"absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100\"\u003e\u003ca class=\"group-hover:text-primary-300 dark:group-hover:text-neutral-700\" style=\"text-decoration-line: none !important;\" href=\"#what-is-a-group\" aria-label=\"Anchor\"\u003e#\u003c/a\u003e\u003c/span\u003e\u003c/h2\u003e\u003cp\u003eA group is a \u003cdel\u003every\u003c/del\u003e abstract mathematical concept that I like to thing of as an extension to\nthe notion of a \u003cstrong\u003eset\u003c/strong\u003e. That is, a set is quite an abstract mathematical concept. A set is an object that\n\u0026ldquo;groups\u0026rdquo; any elements by its properties, or even using any \u0026ldquo;logic\u0026rdquo;\nwe can think of.\u003c/p\u003e\n\u003cp\u003eOne can define a set to be, for example, all people of age 32.\nMathematically, one can define a set of all real numbers, a set of all even numbers, a set of even functions, etc\u0026hellip;\nA set can be finite, countably infinite, uncountably infinite, etc\u0026hellip;\nA set can be caracterized by different things, for example\nthe cardinality of the set. The cardinality can be thought of as\nthe size of the set.\u003c/p\u003e","title":"Applications of group representation theory"},{"content":"GPGPU programming is quite a promising field, involved in different applications. In majority, they are more science related, such as, linear algebra (matrix and tensor operations). Many of these packages are already successfully implemented in ready-to-use libraries.\nI started to study GPGPU programming on my own. And due to the fact that, in my opinion, the CUDA programming is poorly documented (i.e. videos, tutorials, books), I decided then to write my personal notes on this topic. These notes include in fact a significant number of information from different sources, such as books, online tutorials, NVidia\u0026rsquo;s official documentation and blogs.\nNotes are available on my Github repository, together with the LaTeX sources and the .pdf file. They are also available here, at the end of the document\nTopics covered # Introduction Basics of Architecture Programming in CUDA Basic setup on Linux Threads \u0026amp; blocks Memory model Basic algorithms \u0026amp; patterns Reduce CUDA synchronization Introduction to programming tools Notes extract #4. CUDA synchronization mechanisms #As we\u0026rsquo;ve discussed in the section on architecture, when writing kernels, we always need to think about the GPU\u0026rsquo;s hardware, scheduling, etc... By now, with the basic examples we\u0026rsquo;ve considered only basic operations. Indeed, the only API function to synchronize execution, that we\u0026rsquo;ve used in the kernel is the __syncthreads(). This is a simple syncing mechanism, that ensures that all the threads within the block are done before this function invocation.\nCooperative Groups #__device__ int reduce_sum( cooperative_groups::thread_group gr, \\ int *temp, int val){ int lane = g.thread_rank(); for (int i = g.size() / 2; i \u0026gt; 0; i /= 2){ //map each element in the first \u0026#34;semi\u0026#34; block //to it\u0026#39;s corresponding element in the second one temp[lane] = val; g.sync(); // wait for all threads to store if(lane\u0026lt;i) val += temp[lane + i]; g.sync(); // wait for all threads in to load } return val; //only thread 0 will return full sum } Cooperative groups is a relatively new feature to the CUDA API. As the name suggests it, this feature enables us to group threads, with the ability to perform common, collective operations (or simply collectives). We can also perform synchronization between the threads, belonging to the same cooperative groups. With these API features, one can simplify the code, thus avoiding common mistakes and making it more readable. For example, in the code snippet above, we perform the exact same algorithm as in the section on reduce algorithm, by using some utility of the CUDA API. In this case, the cooperative_groups::thread_group class (do not pay attention to how we created this object and/or how it is declared). The thread_rank() function gets the ID/rank of the thread within the thread group g (the same way as threadId.x within a block). Then we\u0026rsquo;re calling the sync() function, which ensures that all the threads within the thread group will be done setting the val, and the second to ensure that all threads are done reducing. It is important to understand, that all the threads will return the val. However, only the thread 0 will accumulate all the val\u0026rsquo;s [10].\nThread blocks #We\u0026rsquo;ve already seen the notion of a thread block many times. This notion was always quite abstract and implicit. Indeed, while launching the kernel, we\u0026rsquo;ve always kept the notion of thread blocks in our mind, but never actually accessed it explicitly. However, in newly introduced features, we can \u0026quot;access\u0026quot; the thread block explicitly. Remember the legacy __syncthreads() function. Well, syncing the this_thread_block() does the same thing as the __syncthreads(). Thus, there are several ways/semantics to synchronize the threads. The following function calls are synonyms.\nauto tb = this_thread_block(); // gets the thread block in kernel tb.sync() // same method as in cooperative_groups cooperative_groups::synchronize(tb); this_thread_block().synchronize(); cooperative_groups::synchronize(this_thread_block()); One can also mention other synonyms dim3 threadIdx $\\equiv$ dim3 thread_index() and dim3 blockIdx $\\equiv$ dim3 group_index(). Thus one can easily replace these built-in keywords with these new methods, without any noticeable performance issues.\nPartitioning #For these cooperative groups, a partitioning feature is also available. For instance, if we\u0026rsquo;ve created a thread thread block, by invoking auto tb = this_thread_block(), one can divide it into more small parts, for instance, into groups of 16 threads. This is done using the cooperative_groups::partition(), method, which takes the subject itself (the one to be partitioned into groups) and the number of threads per group. For instance, cooperative_groups::partition(tb, 16) divides the thread block into groups of 16 threads (so if e.g. a block has a max of 64 threads, this function will create) 4 groups of 16 threads in each.\nThe object returned is a thread_group. By accessing this object, it is possible to get the thread\u0026rsquo;s rank, within the obtained thread_group (for instance, if we divide the thread block into groups of 16 threads and, by passing this object to a device function, print the thread_rank(), method, we will see numbers varying from 0 to 15).\nThe utility of these features overall is to reduce the errors in code. Indeed, the NVidia documentation states that the usage of these features significantly reduces the risk of deadlocks. The concept of deadlocks is probably well-known to the reader. This is a typical situation when we don\u0026rsquo;t want different threads to access a critical section, and ask them to be synchronized before accessing them. Consider the two pieces of code:\n__device__ int sum(int *x, int n) { ... __syncthreads(); ... } __global__ void parallel_kernel(float *x, int n) { if (threadIdx.x \u0026lt; blockDim.x / 2){ sum(x, count); // Half of the threads enter and //the other half-not } } __device__ int sum(thread_block block, int *x, int n) { ... block.sync(); ... } __global__ void parallel_kernel(float *x, int n) { sum(this_thread_block(), x, count); //OK } We clearly see a deadlock in the first piece of code, as there are only threads with ID\u0026rsquo;s less than the half of the block dimension, which will enter the if condition. Those threads will perform their piece of code independently and then will wait for all the other threads in the block to be completed and synchronized. This is a big issue, as there are threads, which will never start the sum() kernel and will just sit up there, waiting for those, who have. So the two chunks of threads will just sit and wait for each other infinitely.\nThis is why one may find an application for the previously discussed primitives. In the second piece of code, one call the method sum() with a thread block. However, we could have divided it into groups, using the CUDA functionality discussed above, and only synchronize that block, which, we are sure, will be run by all threads in the block.\nWarp synchronizations #While programing with CUDA, one never gets tired to think about warps, and how to optimize their execution. I do agree that it is not an easy task, to think of it while doing even some basic operations. The new NVidia architectures and new versions of the CUDA API, provides a simple way to navigate through these concepts.\nWe have discussed a lot of the advantages of shared memory (e.g. for the speed and efficiency of the reduce algorithm). However, the new utilities give us a faster, or even more local way to perform some operations. Remember, shared memory is block-local memory. Remember also that every thread has some kind of register to store small intermediate values while performing a kernel, for example, when we used to store the local thread ID. The so-called warp level synchronization primitives allow us to access a certain thread\u0026rsquo;s local register from an-another thread, [as long as they are in the same warp]{.underline}, without the usage of the shared memory. Again, there are many things to keep in mind, but if such a function is called, it is doing everything atomically, in the sense that it is a primitive operation, performed locally on the threads in the warp. We therefore introduce here the notion of the lane (in fact, we used it briefly above). A lane is the thread id within the warp.\nA little disclaimer: #There are various Warp-level primitive functions. We will note that many function\u0026rsquo;s names are similar, and only differ by the postfix _sync(). For instance, __shfl_xor() and __shfl_xor_sync() [3], [7]. Indeed, the ones with the _sync() postfix are an improvement of the former. It is recommended to use the newer version instead. I will not go into great detail about these differences. I will just mention that there are differences in parameters [^18](see further examples).\nThe __activemask() primitive/function is actually not a synchronization mechanism, but more of filtering mechanisms. This function returns the indices of the active threads in the warp, where it was referenced from[^19]. So the result returned from the __activemask() is used to call other synchronization functions, to give them the corresponding threads, that are active in the warp. So, for example, one could call the __syncwarp(MASK), thus asking to sync all the threads meeting the __activemask() condition. One can go to the NVidia\u0026rsquo;s developer\u0026rsquo;s guide [2] and find the following:\nReturns a 32-bit integer mask of all currently active threads in the calling warp. The Nth bit is set if the Nth lane in the warp is active when __activemask() is called. Inactive threads are represented by 0 bits in the returned mask.\nSo let\u0026rsquo;s have a quick look and example of what did NVidia provide us with [^20]\n__shfl_sync(), or, in previous CUDA versions, __shfl() is a tool, to \u0026quot;broadcast\u0026quot;, or \u0026quot;spread\u0026quot; a certain value from a certain thread (identified with its lane) to all others in the warp. For example, in a certain warp, all the threads have a variable int b = //some random int, unique for all the threads. And I want all these thread\u0026rsquo;s variable b, to be the same as the one in the thread 4 (its lane number or ID within the warp). The best way to do that is to use the provided function: __shfl_sync(0xffffffff, b, 4) (or __shfl(b,4)). The second parameter is the variable to be broadcasted and the third one is the lane number to take the value from (because, of course, all the threads have this local variable b, which is different for all of them). So we\u0026rsquo;re replacing all the b\u0026rsquo;s with THE b of the thread 4. The first parameter is actually the mask/filter, that tells the processor (or core I should say) which threads will be involved in this operation. This can be used by passing the result of e.g. __activemask() function (there are multiple filtering functions) or by passing it the default value in hex notation, which corresponds to the maximum number that can be displayed in binary notation (all the 32 bits are 1, thus we\u0026rsquo;re saying that all the threads are active).\n__shfl_up_sync() or in previous CUDA versions __shfl_up() is a function to shift the values of the warp by an offset. For instance, let\u0026rsquo;s say that in the warp, I want the 4\u0026rsquo;rd thread to have the value from 0\u0026rsquo;th thread, the 5\u0026rsquo;th thread the value of the 1\u0026rsquo;st, the 6\u0026rsquo;th thread the value from the 2\u0026rsquo;nd thread, etc ... Then we would want to use the __shfl_up_sync() function, with the same parameters as in the __shfl_sync() function described above.\n__shfl_down_sync() Is the same idea as the __shfl_up_sync(). The difference is that we would use it if we wanted e.g. the thread 29 to have the value 31. (see figure for better understanding).\nThese primitives come in various shapes and forms. It would take quite a time to discuss them all here. The idea for them all, however, follows quite well the patterns, we\u0026rsquo;ve discussed just above. It is important to understand that these operations are sort of atomic, because, as we\u0026rsquo;ve seen, these are warp-local primitives, which is the most fundamental part of the execution scheduling model.\nThe full pdf # ","date":"February 22, 2023","permalink":"https://blog.leokrglv.net/posts/cuda_notes/","section":"Posts","summary":"\u003cp\u003eGPGPU programming is quite a promising field, involved in different applications. In majority, they are more \u003cem\u003escience related\u003c/em\u003e, such as, linear algebra (matrix and tensor operations). Many of these packages\nare already successfully implemented in \u003cem\u003eready-to-use\u003c/em\u003e libraries.\u003c/p\u003e\n\u003cp\u003eI started to study GPGPU programming on my own. And due to the fact that, in my opinion, the CUDA programming is poorly documented (i.e. videos, tutorials, books), I decided then to write\nmy personal notes on this topic. These notes include in fact a significant number of information from different sources, such as books, online tutorials, NVidia\u0026rsquo;s official documentation and blogs.\u003c/p\u003e","title":"Cuda Notes"},{"content":"","date":null,"permalink":"https://blog.leokrglv.net/categories/","section":"Categories","summary":"","title":"Categories"},{"content":"","date":null,"permalink":"https://blog.leokrglv.net/tags/","section":"Tags","summary":"","title":"Tags"}]