{"entry_id": "92a8db0d-74b9-4c8a-9678-d1b5453e5e0e", "infos": [{"txt": "Nvidia GPU on bare metal NixOS Kubernetes cluster explained", "language": "english", "position": {"atoms": [{"position_id": 0, "txt": "Nvidia GPU on bare metal NixOS Kubernetes cluster explained", "x": "//title"}]}, "tags": [], "label": "article_title", "web_segment_id": 0, "global_sentence_id": 0, "edu_l1_label": "BOT"}, {"txt": "Since the last time I published the second MAZE (Massive Argumented Zonal Environments) article, I realized that the framework is getting more mature, but I need a solution to run it on a large scale.", "language": "english", "position": {"atoms": [{"position_id": 950, "txt": "Since the last time I published ", "x": "/html/body/div/div/div/article/p[1]"}, {"position_id": 952, "txt": "the second MAZE (Massive Argumented Zonal Environments) article", "x": "/html/body/div/div/div/article/p[1]/a[1]"}, {"position_id": 954, "txt": ", I realized that the framework is getting more mature, but I need a solution to run it on a large scale.", "x": "/html/body/div/div/div/article/p[1]/a[1]"}]}, "tags": [], "label": "content", "web_segment_id": 1, "global_sentence_id": 1, "edu_l1_label": "BOS"}, {"txt": " In the past, I built a bare metal Kubernetes cluster on top of a three-node mini PC interconnected with USB4.", "language": "english", "position": {"atoms": [{"position_id": 955, "txt": " In the past, I built ", "x": "/html/body/div/div/div/article/p[1]/a[1]"}, {"position_id": 957, "txt": "a bare metal Kubernetes cluster on top of a three-node mini PC interconnected with USB4", "x": "/html/body/div/div/div/article/p[1]/a[2]"}, {"position_id": 959, "txt": ".", "x": "/html/body/div/div/div/article/p[1]/a[2]"}]}, "tags": [], "label": "content", "web_segment_id": 1, "global_sentence_id": 2, "edu_l1_label": "IOS"}, {"txt": " I have a retired workstation PC with NVIDIA GeForce RTX 2080 Ti.", "language": "english", "position": {"atoms": [{"position_id": 960, "txt": " I have a retired workstation PC with NVIDIA GeForce RTX 2080 Ti.", "x": "/html/body/div/div/div/article/p[1]/a[2]"}]}, "tags": [], "label": "content", "web_segment_id": 1, "global_sentence_id": 3, "edu_l1_label": "IOS"}, {"txt": " I wondered why not put this PC into the three-node Kubernetes cluster and configure Kubernetes to make CUDA work.", "language": "english", "position": {"atoms": [{"position_id": 961, "txt": " I wondered why not put this PC into the three-node Kubernetes cluster and configure Kubernetes to make CUDA work.", "x": "/html/body/div/div/div/article/p[1]/a[2]"}]}, "tags": [], "label": "content", "web_segment_id": 1, "global_sentence_id": 4, "edu_l1_label": "IOS"}, {"txt": " Once I have one configured, extending the computing power capacity with more GPUs will be easier.", "language": "english", "position": {"atoms": [{"position_id": 962, "txt": " Once I have one configured, extending the computing power capacity with more GPUs will be easier.", "x": "/html/body/div/div/div/article/p[1]/a[2]"}]}, "tags": [], "label": "content", "web_segment_id": 1, "global_sentence_id": 5, "edu_l1_label": "IOS"}, {"txt": "<img img", "language": "english", "position": {"atoms": [{"position_id": 51, "txt": "<img img", "x": "/html/body/div/div/div/article/figure[1]/img"}]}, "tags": ["img"], "label": "figure", "web_segment_id": 2, "global_sentence_id": 6, "edu_l1_label": "EDU_O"}, {"txt": "Screenshot of btop of my four-node Kubernetes cluster, with three mini PCs connected via USB4 and the newly added retired workstation sporting an Nvidia GeForce RTX 2080 Ti.", "language": "english", "position": {"atoms": [{"position_id": 964, "txt": "Screenshot of ", "x": "/html/body/div/div/div/article/figure[1]/figcaption/p"}, {"position_id": 966, "txt": "btop", "x": "/html/body/div/div/div/article/figure[1]/figcaption/p/a"}, {"position_id": 968, "txt": " of my four-node Kubernetes cluster, with three mini PCs connected via USB4 and the newly added retired workstation sporting an Nvidia GeForce RTX 2080 Ti.", "x": "/html/body/div/div/div/article/figure[1]/figcaption/p/a"}]}, "tags": ["img", "figcaption"], "label": "figure_title", "web_segment_id": 3, "global_sentence_id": 7, "edu_l1_label": "EDU_O"}, {"txt": "With that in mind, I took a week on this side quest.", "language": "english", "position": {"atoms": [{"position_id": 970, "txt": "With that in mind, I took a week on this side quest.", "x": "/html/body/div/div/div/article/p[2]"}]}, "tags": [], "label": "content", "web_segment_id": 4, "global_sentence_id": 8, "edu_l1_label": "IOS"}, {"txt": " It turns out it’s way harder than I expected.", "language": "english", "position": {"atoms": [{"position_id": 971, "txt": " It turns out it’s way harder than I expected.", "x": "/html/body/div/div/div/article/p[2]"}]}, "tags": [], "label": "content", "web_segment_id": 4, "global_sentence_id": 9, "edu_l1_label": "IOS"}, {"txt": " I have learned a lot from digging this rabbit hole, and I encountered other holes while digging this one.", "language": "english", "position": {"atoms": [{"position_id": 972, "txt": " I have learned a lot from digging this rabbit hole, and I encountered other holes while digging this one.", "x": "/html/body/div/div/div/article/p[2]"}]}, "tags": [], "label": "content", "web_segment_id": 4, "global_sentence_id": 10, "edu_l1_label": "IOS"}, {"txt": " Getting the Nvidia device plugin up and running in Kubernetes is hard.", "language": "english", "position": {"atoms": [{"position_id": 973, "txt": " Getting the ", "x": "/html/body/div/div/div/article/p[2]"}, {"position_id": 975, "txt": "Nvidia device plugin", "x": "/html/body/div/div/div/article/p[2]/a[1]"}, {"position_id": 977, "txt": " up and running in Kubernetes is hard.", "x": "/html/body/div/div/div/article/p[2]/a[1]"}]}, "tags": [], "label": "content", "web_segment_id": 4, "global_sentence_id": 11, "edu_l1_label": "IOS"}, {"txt": " Not to mention, running on top of NixOS makes it even more challenging.", "language": "english", "position": {"atoms": [{"position_id": 978, "txt": " Not to mention, running on top of ", "x": "/html/body/div/div/div/article/p[2]/a[1]"}, {"position_id": 980, "txt": "NixOS", "x": "/html/body/div/div/div/article/p[2]/a[2]"}, {"position_id": 982, "txt": " makes it even more challenging.", "x": "/html/body/div/div/div/article/p[2]/a[2]"}]}, "tags": [], "label": "content", "web_segment_id": 4, "global_sentence_id": 12, "edu_l1_label": "IOS"}, {"txt": " Regardless, I finally managed to get it to work!", "language": "english", "position": {"atoms": [{"position_id": 983, "txt": " Regardless, I finally managed to get it to work!", "x": "/html/body/div/div/div/article/p[2]/a[2]"}]}, "tags": [], "label": "content", "web_segment_id": 4, "global_sentence_id": 13, "edu_l1_label": "IOS"}, {"txt": " Seeing it running for the first time was a very joyful moment.", "language": "english", "position": {"atoms": [{"position_id": 984, "txt": " Seeing it running for the first time was a very joyful moment.", "x": "/html/body/div/div/div/article/p[2]/a[2]"}]}, "tags": [], "label": "content", "web_segment_id": 4, "global_sentence_id": 14, "edu_l1_label": "IOS"}, {"txt": "<img img", "language": "english", "position": {"atoms": [{"position_id": 59, "txt": "<img img", "x": "/html/body/div/div/div/article/figure[2]/img"}]}, "tags": ["img"], "label": "figure", "web_segment_id": 2, "global_sentence_id": 15, "edu_l1_label": "EDU_O"}, {"txt": "Screenshot of the Kubernetes dashboard shows the logs of a MAZE pod running an experiment on a CUDA device", "language": "english", "position": {"atoms": [{"position_id": 986, "txt": "Screenshot of the ", "x": "/html/body/div/div/div/article/figure[2]/figcaption/p"}, {"position_id": 988, "txt": "Kubernetes dashboard", "x": "/html/body/div/div/div/article/figure[2]/figcaption/p/a"}, {"position_id": 990, "txt": " shows the logs of a MAZE pod running an experiment on a CUDA device", "x": "/html/body/div/div/div/article/figure[2]/figcaption/p/a"}]}, "tags": ["figcaption", "img"], "label": "figure_title", "web_segment_id": 5, "global_sentence_id": 16, "edu_l1_label": "EDU_O"}, {"txt": "An article like this could be helpful to someone as more software engineers are getting into machine learning.", "language": "english", "position": {"atoms": [{"position_id": 992, "txt": "An article like this could be helpful to someone as more software engineers are getting into machine learning.", "x": "/html/body/div/div/div/article/p[3]"}]}, "tags": [], "label": "content", "web_segment_id": 6, "global_sentence_id": 17, "edu_l1_label": "IOS"}, {"txt": " Running all of it on the cloud is a costly option.", "language": "english", "position": {"atoms": [{"position_id": 993, "txt": " Running all of it on the cloud is a costly option.", "x": "/html/body/div/div/div/article/p[3]"}]}, "tags": [], "label": "content", "web_segment_id": 6, "global_sentence_id": 18, "edu_l1_label": "IOS"}, {"txt": " There are also privacy concerns for personal use, so Nvidia GPU on a local bare-metal Kubernetes cluster is still a very tempting option.", "language": "english", "position": {"atoms": [{"position_id": 994, "txt": " There are also privacy concerns for personal use, so Nvidia GPU on a local bare-metal Kubernetes cluster is still a very tempting option.", "x": "/html/body/div/div/div/article/p[3]"}]}, "tags": [], "label": "content", "web_segment_id": 6, "global_sentence_id": 19, "edu_l1_label": "IOS"}, {"txt": " Here, I would like to share my experience setting up a bare metal Kubernetes cluster with the Nvidia GPU CDI plugin enabled in NixOS.", "language": "english", "position": {"atoms": [{"position_id": 995, "txt": " Here, I would like to share my experience setting up a bare metal Kubernetes cluster with the Nvidia GPU CDI plugin enabled in NixOS.", "x": "/html/body/div/div/div/article/p[3]"}]}, "tags": [], "label": "content", "web_segment_id": 6, "global_sentence_id": 20, "edu_l1_label": "IOS"}, {"txt": "Chain of rabbit holes", "language": "english", "position": {"atoms": [{"position_id": 997, "txt": "Chain of rabbit holes", "x": "/html/body/div/div/div/article/h2[1]"}]}, "tags": ["h2"], "label": "title1", "web_segment_id": 7, "global_sentence_id": 21, "edu_l1_label": "BOS"}, {"txt": "I want to make my Kubernetes support Nvidia GPU, not because I like it.", "language": "english", "position": {"atoms": [{"position_id": 999, "txt": "I want to make my Kubernetes support Nvidia GPU, not because I like it.", "x": "/html/body/div/div/div/article/p[4]"}]}, "tags": [], "label": "content", "web_segment_id": 8, "global_sentence_id": 22, "edu_l1_label": "IOS"}, {"txt": " I just need to find a way to scale MAZE to continue my research more efficiently.", "language": "english", "position": {"atoms": [{"position_id": 1000, "txt": " I just need to find a way to scale MAZE to continue my research more efficiently.", "x": "/html/body/div/div/div/article/p[4]"}]}, "tags": [], "label": "content", "web_segment_id": 8, "global_sentence_id": 23, "edu_l1_label": "IOS"}, {"txt": " In software engineering, when you encounter a problem that requires dedicated time and effort to find the solution or root cause, I call it digging a rabbit hole.", "language": "english", "position": {"atoms": [{"position_id": 1001, "txt": " In software engineering, when you encounter a problem that requires dedicated time and effort to find the solution or root cause, I call it digging a rabbit hole.", "x": "/html/body/div/div/div/article/p[4]"}]}, "tags": [], "label": "content", "web_segment_id": 8, "global_sentence_id": 24, "edu_l1_label": "IOS"}, {"txt": " It’s a reference from Alice’s Adventures in Wonderland’s Down the rabbit hole.", "language": "english", "position": {"atoms": [{"position_id": 1002, "txt": " It’s a reference from ", "x": "/html/body/div/div/div/article/p[4]"}, {"position_id": 1004, "txt": "Alice’s Adventures in Wonderland", "x": "/html/body/div/div/div/article/p[4]/a[1]"}, {"position_id": 1006, "txt": "’s ", "x": "/html/body/div/div/div/article/p[4]/a[1]"}, {"position_id": 1008, "txt": "Down the rabbit hole", "x": "/html/body/div/div/div/article/p[4]/a[2]"}, {"position_id": 1010, "txt": ".", "x": "/html/body/div/div/div/article/p[4]/a[2]"}]}, "tags": [], "label": "content", "web_segment_id": 8, "global_sentence_id": 25, "edu_l1_label": "IOS"}, {"txt": "<img img", "language": "english", "position": {"atoms": [{"position_id": 69, "txt": "<img img", "x": "/html/body/div/div/div/article/figure[3]/img"}]}, "tags": ["img"], "label": "figure", "web_segment_id": 2, "global_sentence_id": 26, "edu_l1_label": "EDU_O"}, {"txt": "Rabbit from Alice's Adventures in Wonderland, by John Tenniel", "language": "english", "position": {"atoms": [{"position_id": 1012, "txt": "Rabbit from Alice's Adventures in Wonderland, by ", "x": "/html/body/div/div/div/article/figure[3]/figcaption/p"}, {"position_id": 1014, "txt": "John Tenniel", "x": "/html/body/div/div/div/article/figure[3]/figcaption/p/a"}]}, "tags": ["figcaption", "img"], "label": "figure_title", "web_segment_id": 9, "global_sentence_id": 27, "edu_l1_label": "EDU_O"}, {"txt": "Interestingly, during the process of digging a rabbit hole, you would often find yourself needing to dig another due to a new obstacle.", "language": "english", "position": {"atoms": [{"position_id": 1016, "txt": "Interestingly, during the process of digging a rabbit hole, you would often find yourself needing to dig another due to a new obstacle.", "x": "/html/body/div/div/div/article/p[5]"}]}, "tags": [], "label": "content", "web_segment_id": 10, "global_sentence_id": 28, "edu_l1_label": "IOS"}, {"txt": " I call it the chain of rabbit holes.", "language": "english", "position": {"atoms": [{"position_id": 1017, "txt": " I call it the chain of rabbit holes.", "x": "/html/body/div/div/div/article/p[5]"}]}, "tags": [], "label": "content", "web_segment_id": 10, "global_sentence_id": 29, "edu_l1_label": "IOS"}, {"txt": " For example, I just heard in the All-in podcast they mentioned that for Elon Musk to build Gork in a short time frame, they had to", "language": "english", "position": {"atoms": [{"position_id": 1018, "txt": " For example, I just heard in the ", "x": "/html/body/div/div/div/article/p[5]"}, {"position_id": 1020, "txt": "All-in podcast they mentioned", "x": "/html/body/div/div/div/article/p[5]/a"}, {"position_id": 1022, "txt": " that for Elon Musk to build Gork in a short time frame, they had to", "x": "/html/body/div/div/div/article/p[5]/a"}]}, "tags": [], "label": "content", "web_segment_id": 10, "global_sentence_id": 30, "edu_l1_label": "IOS"}, {"txt": "Find a place for putting one hundred GPU", "language": "english", "position": {"atoms": [{"position_id": 1024, "txt": "Find a place for putting one hundred GPU", "x": "/html/body/div/div/div/article/ol/li[1]"}]}, "tags": ["li"], "label": "content", "web_segment_id": 11, "global_sentence_id": 31, "edu_l1_label": "IOS"}, {"txt": "Need power, so buy generators", "language": "english", "position": {"atoms": [{"position_id": 1026, "txt": "Need power, so buy generators", "x": "/html/body/div/div/div/article/ol/li[2]"}]}, "tags": ["li"], "label": "content", "web_segment_id": 12, "global_sentence_id": 32, "edu_l1_label": "IOS"}, {"txt": "Need cooling, so buy portable liquid cooling", "language": "english", "position": {"atoms": [{"position_id": 1028, "txt": "Need cooling, so buy portable liquid cooling", "x": "/html/body/div/div/div/article/ol/li[3]"}]}, "tags": ["li"], "label": "content", "web_segment_id": 13, "global_sentence_id": 33, "edu_l1_label": "IOS"}, {"txt": "Realize the need to smooth power usage, so install the Tesla powerpack", "language": "english", "position": {"atoms": [{"position_id": 1030, "txt": "Realize the need to smooth power usage, so install the Tesla powerpack", "x": "/html/body/div/div/div/article/ol/li[4]"}]}, "tags": ["li"], "label": "content", "web_segment_id": 14, "global_sentence_id": 34, "edu_l1_label": "IOS"}, {"txt": "…", "language": "english", "position": {"atoms": [{"position_id": 1032, "txt": "…", "x": "/html/body/div/div/div/article/ol/li[5]"}]}, "tags": ["li"], "label": "content", "web_segment_id": 15, "global_sentence_id": 35, "edu_l1_label": "IOS"}, {"txt": "This is a great example of a chain of rabbit holes.", "language": "english", "position": {"atoms": [{"position_id": 1034, "txt": "This is a great example of a chain of rabbit holes.", "x": "/html/body/div/div/div/article/p[6]"}]}, "tags": [], "label": "content", "web_segment_id": 16, "global_sentence_id": 36, "edu_l1_label": "IOS"}, {"txt": " The ability to dig rabbit holes and solve hard problems down the chain boils down to two words:", "language": "english", "position": {"atoms": [{"position_id": 1035, "txt": " The ability to dig rabbit holes and solve hard problems down the chain boils down to two words:", "x": "/html/body/div/div/div/article/p[6]"}]}, "tags": [], "label": "content", "web_segment_id": 16, "global_sentence_id": 37, "edu_l1_label": "IOS"}, {"txt": "Execution power", "language": "english", "position": {"atoms": [{"position_id": 1037, "txt": "Execution power", "x": "/html/body/div/div/div/article/blockquote[1]/p"}]}, "tags": [], "label": "content", "web_segment_id": 17, "global_sentence_id": 38, "edu_l1_label": "IOS"}, {"txt": "The reason for Deepseek’s recent successful story also boils down to execution power.", "language": "english", "position": {"atoms": [{"position_id": 1039, "txt": "The reason for Deepseek’s recent successful story also boils down to execution power.", "x": "/html/body/div/div/div/article/p[7]"}]}, "tags": [], "label": "content", "web_segment_id": 18, "global_sentence_id": 39, "edu_l1_label": "IOS"}, {"txt": " They recently announced their customized distributed file system to optimize AI training.", "language": "english", "position": {"atoms": [{"position_id": 1040, "txt": " They recently announced their ", "x": "/html/body/div/div/div/article/p[7]"}, {"position_id": 1042, "txt": "customized distributed file system to optimize AI training", "x": "/html/body/div/div/div/article/p[7]/a[1]"}, {"position_id": 1044, "txt": ".", "x": "/html/body/div/div/div/article/p[7]/a[1]"}]}, "tags": [], "label": "content", "web_segment_id": 18, "global_sentence_id": 40, "edu_l1_label": "IOS"}, {"txt": " Their ability to dig rabbit holes like this or the previous PTX (Parallel Thread Execution) optimization shows great execution power.", "language": "english", "position": {"atoms": [{"position_id": 1045, "txt": " Their ability to dig rabbit holes like this or the previous ", "x": "/html/body/div/div/div/article/p[7]/a[1]"}, {"position_id": 1047, "txt": "PTX (Parallel Thread Execution)", "x": "/html/body/div/div/div/article/p[7]/a[2]"}, {"position_id": 1049, "txt": " optimization shows great execution power.", "x": "/html/body/div/div/div/article/p[7]/a[2]"}]}, "tags": [], "label": "content", "web_segment_id": 18, "global_sentence_id": 41, "edu_l1_label": "IOS"}, {"txt": "It may sound obvious, but there are many interesting aspects to consider while digging rabbit holes.", "language": "english", "position": {"atoms": [{"position_id": 1051, "txt": "It may sound obvious, but there are many interesting aspects to consider while digging rabbit holes.", "x": "/html/body/div/div/div/article/p[8]"}]}, "tags": [], "label": "content", "web_segment_id": 19, "global_sentence_id": 42, "edu_l1_label": "IOS"}, {"txt": " Of course, with constraints, one cannot dig all the rabbit holes, and it would take forever to finish the project.", "language": "english", "position": {"atoms": [{"position_id": 1052, "txt": " Of course, with constraints, one cannot dig all the rabbit holes, and it would take forever to finish the project.", "x": "/html/body/div/div/div/article/p[8]"}]}, "tags": [], "label": "content", "web_segment_id": 19, "global_sentence_id": 43, "edu_l1_label": "IOS"}, {"txt": " Also, not all rabbit holes are worth digging.", "language": "english", "position": {"atoms": [{"position_id": 1053, "txt": " Also, not all rabbit holes are worth digging.", "x": "/html/body/div/div/div/article/p[8]"}]}, "tags": [], "label": "content", "web_segment_id": 19, "global_sentence_id": 44, "edu_l1_label": "IOS"}, {"txt": " Since this is an interesting topic but rarely mentioned, I would bring up a bit of my thought process on my decision going down each rabbit hole.", "language": "english", "position": {"atoms": [{"position_id": 1054, "txt": " Since this is an interesting topic but rarely mentioned, I would bring up a bit of my thought process on my decision going down each rabbit hole.", "x": "/html/body/div/div/div/article/p[8]"}]}, "tags": [], "label": "content", "web_segment_id": 19, "global_sentence_id": 45, "edu_l1_label": "IOS"}, {"txt": " Before that, let’s see the chain of rabbit holes I was digging in the past week:", "language": "english", "position": {"atoms": [{"position_id": 1055, "txt": " Before that, let’s see the chain of rabbit holes I was digging in the past week:", "x": "/html/body/div/div/div/article/p[8]"}]}, "tags": [], "label": "content", "web_segment_id": 19, "global_sentence_id": 46, "edu_l1_label": "IOS"}, {"txt": "<img img", "language": "english", "position": {"atoms": [{"position_id": 89, "txt": "<img img", "x": "/html/body/div/div/div/article/figure[4]/img"}]}, "tags": ["img"], "label": "figure", "web_segment_id": 2, "global_sentence_id": 47, "edu_l1_label": "EDU_O"}, {"txt": "Diagram illustrating the chain of rabbit holes I encountered while setting up Nvidia GPU support on my Kubernetes cluster.", "language": "english", "position": {"atoms": [{"position_id": 1057, "txt": "Diagram illustrating the chain of rabbit holes I encountered while setting up Nvidia GPU support on my Kubernetes cluster.", "x": "/html/body/div/div/div/article/figure[4]/figcaption/p"}]}, "tags": ["figcaption", "img"], "label": "figure_title", "web_segment_id": 20, "global_sentence_id": 48, "edu_l1_label": "EDU_O"}, {"txt": "Running Kubernetes with NixOS", "language": "english", "position": {"atoms": [{"position_id": 1059, "txt": "Running Kubernetes with NixOS", "x": "/html/body/div/div/div/article/h2[2]"}]}, "tags": ["h2"], "label": "title1", "web_segment_id": 21, "global_sentence_id": 49, "edu_l1_label": "BOS"}, {"txt": "It is not a new rabbit hole I dug.", "language": "english", "position": {"atoms": [{"position_id": 1061, "txt": "It is not a new rabbit hole I dug.", "x": "/html/body/div/div/div/article/p[9]"}]}, "tags": [], "label": "content", "web_segment_id": 22, "global_sentence_id": 50, "edu_l1_label": "IOS"}, {"txt": " I have already figured out how to deploy Kubernetes on NixOS as needed while building the three-node cluster.", "language": "english", "position": {"atoms": [{"position_id": 1062, "txt": " I have already figured out how to deploy Kubernetes on NixOS as needed while building the three-node cluster.", "x": "/html/body/div/div/div/article/p[9]"}]}, "tags": [], "label": "content", "web_segment_id": 22, "global_sentence_id": 51, "edu_l1_label": "IOS"}, {"txt": " But since I didn’t mention details in the previous article, bringing up the details here makes sense.", "language": "english", "position": {"atoms": [{"position_id": 1063, "txt": " But since I didn’t mention details in the previous article, bringing up the details here makes sense.", "x": "/html/body/div/div/div/article/p[9]"}]}, "tags": [], "label": "content", "web_segment_id": 22, "global_sentence_id": 52, "edu_l1_label": "IOS"}, {"txt": "I love NixOS and Nixpkgs.", "language": "english", "position": {"atoms": [{"position_id": 1065, "txt": "I love NixOS and ", "x": "/html/body/div/div/div/article/p[10]"}, {"position_id": 1067, "txt": "Nixpkgs", "x": "/html/body/div/div/div/article/p[10]/a"}, {"position_id": 1069, "txt": ".", "x": "/html/body/div/div/div/article/p[10]/a"}]}, "tags": [], "label": "content", "web_segment_id": 23, "global_sentence_id": 53, "edu_l1_label": "IOS"}, {"txt": " The ability to configure a whole operation system with just configuration files, making it reproducible, is simply amazing.", "language": "english", "position": {"atoms": [{"position_id": 1070, "txt": " The ability to configure a whole operation system with just configuration files, making it reproducible, is simply amazing.", "x": "/html/body/div/div/div/article/p[10]/a"}]}, "tags": [], "label": "content", "web_segment_id": 23, "global_sentence_id": 54, "edu_l1_label": "IOS"}, {"txt": " Here’s an example of a hello world NixOS config file:", "language": "english", "position": {"atoms": [{"position_id": 1071, "txt": " Here’s an example of a hello world NixOS config file:", "x": "/html/body/div/div/div/article/p[10]/a"}]}, "tags": [], "label": "content", "web_segment_id": 23, "global_sentence_id": 55, "edu_l1_label": "IOS"}, {"txt": "{config,lib,pkgs,...}:", "language": "english", "position": {"atoms": [{"position_id": 1073, "txt": "{", "x": "/html/body/div/div/div/article/div[1]/div/pre/code/span[1]"}, {"position_id": 1075, "txt": "config", "x": "/html/body/div/div/div/article/div[1]/div/pre/code/span[2]"}, {"position_id": 1077, "txt": ",", "x": "/html/body/div/div/div/article/div[1]/div/pre/code/span[3]"}, {"position_id": 1079, "txt": "lib", "x": "/html/body/div/div/div/article/div[1]/div/pre/code/span[4]"}, {"position_id": 1081, "txt": ",", "x": "/html/body/div/div/div/article/div[1]/div/pre/code/span[5]"}, {"position_id": 1083, "txt": "pkgs", "x": "/html/body/div/div/div/article/div[1]/div/pre/code/span[6]"}, {"position_id": 1085, "txt": ",", "x": "/html/body/div/div/div/article/div[1]/div/pre/code/span[7]"}, {"position_id": 1087, "txt": "...", "x": "/html/body/div/div/div/article/div[1]/div/pre/code/span[8]"}, {"position_id": 1089, "txt": "}:", "x": "/html/body/div/div/div/article/div[1]/div/pre/code/span[9]"}]}, "tags": [], "label": "code", "web_segment_id": 24, "global_sentence_id": 56, "edu_l1_label": "EDU_O"}, {"txt": "{imports=[# Include the results of the hardware scan../hardware-configuration.nix];# Use the systemd-boot EFI boot loader.boot.loader.systemd-boot.enable=true;boot.loader.efi.canTouchEfiVariables=true;system.stateVersion=\"24.11\";}", "language": "english", "position": {"atoms": [{"position_id": 1091, "txt": "{", "x": "/html/body/div/div/div/article/div[1]/div/pre/code/span[10]"}, {"position_id": 1093, "txt": "imports", "x": "/html/body/div/div/div/article/div[1]/div/pre/code/span[11]"}, {"position_id": 1095, "txt": "=", "x": "/html/body/div/div/div/article/div[1]/div/pre/code/span[12]"}, {"position_id": 1097, "txt": "[", "x": "/html/body/div/div/div/article/div[1]/div/pre/code/span[13]"}, {"position_id": 1099, "txt": "# Include the results of the hardware scan.", "x": "/html/body/div/div/div/article/div[1]/div/pre/code/span[14]"}, {"position_id": 1101, "txt": "./hardware-configuration.nix", "x": "/html/body/div/div/div/article/div[1]/div/pre/code/span[15]"}, {"position_id": 1103, "txt": "];", "x": "/html/body/div/div/div/article/div[1]/div/pre/code/span[16]"}, {"position_id": 1105, "txt": "# Use the systemd-boot EFI boot loader.", "x": "/html/body/div/div/div/article/div[1]/div/pre/code/span[17]"}, {"position_id": 1107, "txt": "boot", "x": "/html/body/div/div/div/article/div[1]/div/pre/code/span[18]"}, {"position_id": 1109, "txt": ".", "x": "/html/body/div/div/div/article/div[1]/div/pre/code/span[19]"}, {"position_id": 1111, "txt": "loader", "x": "/html/body/div/div/div/article/div[1]/div/pre/code/span[20]"}, {"position_id": 1113, "txt": ".", "x": "/html/body/div/div/div/article/div[1]/div/pre/code/span[21]"}, {"position_id": 1115, "txt": "systemd-boot", "x": "/html/body/div/div/div/article/div[1]/div/pre/code/span[22]"}, {"position_id": 1117, "txt": ".", "x": "/html/body/div/div/div/article/div[1]/div/pre/code/span[23]"}, {"position_id": 1119, "txt": "enable", "x": "/html/body/div/div/div/article/div[1]/div/pre/code/span[24]"}, {"position_id": 1121, "txt": "=", "x": "/html/body/div/div/div/article/div[1]/div/pre/code/span[25]"}, {"position_id": 1123, "txt": "true", "x": "/html/body/div/div/div/article/div[1]/div/pre/code/span[26]"}, {"position_id": 1125, "txt": ";", "x": "/html/body/div/div/div/article/div[1]/div/pre/code/span[27]"}, {"position_id": 1127, "txt": "boot", "x": "/html/body/div/div/div/article/div[1]/div/pre/code/span[28]"}, {"position_id": 1129, "txt": ".", "x": "/html/body/div/div/div/article/div[1]/div/pre/code/span[29]"}, {"position_id": 1131, "txt": "loader", "x": "/html/body/div/div/div/article/div[1]/div/pre/code/span[30]"}, {"position_id": 1133, "txt": ".", "x": "/html/body/div/div/div/article/div[1]/div/pre/code/span[31]"}, {"position_id": 1135, "txt": "efi", "x": "/html/body/div/div/div/article/div[1]/div/pre/code/span[32]"}, {"position_id": 1137, "txt": ".", "x": "/html/body/div/div/div/article/div[1]/div/pre/code/span[33]"}, {"position_id": 1139, "txt": "canTouchEfiVariables", "x": "/html/body/div/div/div/article/div[1]/div/pre/code/span[34]"}, {"position_id": 1141, "txt": "=", "x": "/html/body/div/div/div/article/div[1]/div/pre/code/span[35]"}, {"position_id": 1143, "txt": "true", "x": "/html/body/div/div/div/article/div[1]/div/pre/code/span[36]"}, {"position_id": 1145, "txt": ";", "x": "/html/body/div/div/div/article/div[1]/div/pre/code/span[37]"}, {"position_id": 1147, "txt": "system", "x": "/html/body/div/div/div/article/div[1]/div/pre/code/span[38]"}, {"position_id": 1149, "txt": ".", "x": "/html/body/div/div/div/article/div[1]/div/pre/code/span[39]"}, {"position_id": 1151, "txt": "stateVersion", "x": "/html/body/div/div/div/article/div[1]/div/pre/code/span[40]"}, {"position_id": 1153, "txt": "=", "x": "/html/body/div/div/div/article/div[1]/div/pre/code/span[41]"}, {"position_id": 1155, "txt": "\"24.11\"", "x": "/html/body/div/div/div/article/div[1]/div/pre/code/span[42]"}, {"position_id": 1157, "txt": ";", "x": "/html/body/div/div/div/article/div[1]/div/pre/code/span[43]"}, {"position_id": 1159, "txt": "}", "x": "/html/body/div/div/div/article/div[1]/div/pre/code/span[44]"}]}, "tags": [], "label": "code", "web_segment_id": 24, "global_sentence_id": 57, "edu_l1_label": "EDU_O"}, {"txt": "While terrific, it doesn’t come without drawbacks, particularly considering deploying it to multiple machines.", "language": "english", "position": {"atoms": [{"position_id": 1161, "txt": "While terrific, it doesn’t come without drawbacks, particularly considering deploying it to multiple machines.", "x": "/html/body/div/div/div/article/p[11]"}]}, "tags": [], "label": "content", "web_segment_id": 25, "global_sentence_id": 58, "edu_l1_label": "IOS"}, {"txt": " The first problem is that while most configurations would be the same for a machine, many things, such as the hostname, would still be different.", "language": "english", "position": {"atoms": [{"position_id": 1162, "txt": " The first problem is that while most configurations would be the same for a machine, many things, such as the hostname, would still be different.", "x": "/html/body/div/div/div/article/p[11]"}]}, "tags": [], "label": "content", "web_segment_id": 25, "global_sentence_id": 59, "edu_l1_label": "IOS"}, {"txt": " One cannot write a simple configuration and apply it everywhere on all the machines.", "language": "english", "position": {"atoms": [{"position_id": 1163, "txt": " One cannot write a simple configuration and apply it everywhere on all the machines.", "x": "/html/body/div/div/div/article/p[11]"}]}, "tags": [], "label": "content", "web_segment_id": 25, "global_sentence_id": 60, "edu_l1_label": "IOS"}, {"txt": " NixOS generates configuration files in the /etc folder as read-only files with symbolic links to the actual config files in the /nix/store folder.", "language": "english", "position": {"atoms": [{"position_id": 1164, "txt": " NixOS generates configuration files in the ", "x": "/html/body/div/div/div/article/p[11]"}, {"position_id": 1166, "txt": "/etc", "x": "/html/body/div/div/div/article/p[11]/code[1]"}, {"position_id": 1168, "txt": " folder as read-only files with symbolic links to the actual config files in the ", "x": "/html/body/div/div/div/article/p[11]/code[1]"}, {"position_id": 1170, "txt": "/nix/store", "x": "/html/body/div/div/div/article/p[11]/code[2]"}, {"position_id": 1172, "txt": " folder.", "x": "/html/body/div/div/div/article/p[11]/code[2]"}]}, "tags": [], "label": "content", "web_segment_id": 25, "global_sentence_id": 61, "edu_l1_label": "IOS"}, {"txt": "<img img", "language": "english", "position": {"atoms": [{"position_id": 148, "txt": "<img img", "x": "/html/body/div/div/div/article/figure[5]/img"}]}, "tags": ["img"], "label": "figure", "web_segment_id": 2, "global_sentence_id": 62, "edu_l1_label": "EDU_O"}, {"txt": "Screenshot of the output from listing /etc folder shows many symbolic links", "language": "english", "position": {"atoms": [{"position_id": 1174, "txt": "Screenshot of the output from listing /etc folder shows many symbolic links", "x": "/html/body/div/div/div/article/figure[5]/figcaption/p"}]}, "tags": ["figcaption", "img"], "label": "figure_title", "web_segment_id": 26, "global_sentence_id": 63, "edu_l1_label": "EDU_O"}, {"txt": "I found a tool called deploy-rs for deploying to different machines with slightly different configurations.", "language": "english", "position": {"atoms": [{"position_id": 1176, "txt": "I found a tool called ", "x": "/html/body/div/div/div/article/p[12]"}, {"position_id": 1178, "txt": "deploy-rs", "x": "/html/body/div/div/div/article/p[12]/a[1]"}, {"position_id": 1180, "txt": " for deploying to different machines with slightly different configurations.", "x": "/html/body/div/div/div/article/p[12]/a[1]"}]}, "tags": [], "label": "content", "web_segment_id": 27, "global_sentence_id": 64, "edu_l1_label": "IOS"}, {"txt": " It allows you to define configurations for each machine with customization and deploy them easily via SSH.", "language": "english", "position": {"atoms": [{"position_id": 1181, "txt": " It allows you to define configurations for each machine with customization and deploy them easily via SSH.", "x": "/html/body/div/div/div/article/p[12]/a[1]"}]}, "tags": [], "label": "content", "web_segment_id": 27, "global_sentence_id": 65, "edu_l1_label": "IOS"}, {"txt": " Usually, this is good enough with simple stuff like hostnames or Linux configurations.", "language": "english", "position": {"atoms": [{"position_id": 1182, "txt": " Usually, this is good enough with simple stuff like hostnames or Linux configurations.", "x": "/html/body/div/div/div/article/p[12]/a[1]"}]}, "tags": [], "label": "content", "web_segment_id": 27, "global_sentence_id": 66, "edu_l1_label": "IOS"}, {"txt": " However, running a full-blown Kubernetes cluster is yet another story.", "language": "english", "position": {"atoms": [{"position_id": 1183, "txt": " However, running a full-blown Kubernetes cluster is yet another story.", "x": "/html/body/div/div/div/article/p[12]/a[1]"}]}, "tags": [], "label": "content", "web_segment_id": 27, "global_sentence_id": 67, "edu_l1_label": "IOS"}, {"txt": " Just PKI (Public key infrastructure) certification generation and deployment are troublesome.", "language": "english", "position": {"atoms": [{"position_id": 1184, "txt": " Just ", "x": "/html/body/div/div/div/article/p[12]/a[1]"}, {"position_id": 1186, "txt": "PKI (Public key infrastructure)", "x": "/html/body/div/div/div/article/p[12]/a[2]"}, {"position_id": 1188, "txt": " certification generation and deployment are troublesome.", "x": "/html/body/div/div/div/article/p[12]/a[2]"}]}, "tags": [], "label": "content", "web_segment_id": 27, "global_sentence_id": 68, "edu_l1_label": "IOS"}, {"txt": "Generating Kubernetes certificates also implies creating secret keys and deploying them into each node.", "language": "english", "position": {"atoms": [{"position_id": 1190, "txt": "Generating Kubernetes certificates also implies creating secret keys and deploying them into each node.", "x": "/html/body/div/div/div/article/p[13]"}]}, "tags": [], "label": "content", "web_segment_id": 28, "global_sentence_id": 69, "edu_l1_label": "IOS"}, {"txt": " NixOS is very good at making a predictable environment with immutable packages and configuration files, but dealing with centralized secret values distribution is not ideal.", "language": "english", "position": {"atoms": [{"position_id": 1191, "txt": " NixOS is very good at making a predictable environment with immutable packages and configuration files, but dealing with centralized secret values distribution is not ideal.", "x": "/html/body/div/div/div/article/p[13]"}]}, "tags": [], "label": "content", "web_segment_id": 28, "global_sentence_id": 70, "edu_l1_label": "IOS"}, {"txt": " I certainly don’t want to commit my secret key to Git history along with the NixOS configuration file.", "language": "english", "position": {"atoms": [{"position_id": 1192, "txt": " I certainly don’t want to commit my secret key to Git history along with the NixOS configuration file.", "x": "/html/body/div/div/div/article/p[13]"}]}, "tags": [], "label": "content", "web_segment_id": 28, "global_sentence_id": 71, "edu_l1_label": "IOS"}, {"txt": " Therefore, we need another approach to deploy dynamically generated configurations and secret keys to all those machines.", "language": "english", "position": {"atoms": [{"position_id": 1193, "txt": " Therefore, we need another approach to deploy dynamically generated configurations and secret keys to all those machines.", "x": "/html/body/div/div/div/article/p[13]"}]}, "tags": [], "label": "content", "web_segment_id": 28, "global_sentence_id": 72, "edu_l1_label": "IOS"}, {"txt": "I am pretty familiar with Ansible because I used it a lot in the past and even contributed to the omit filter many years ago.", "language": "english", "position": {"atoms": [{"position_id": 1195, "txt": "I am pretty familiar with ", "x": "/html/body/div/div/div/article/p[14]"}, {"position_id": 1197, "txt": "Ansible", "x": "/html/body/div/div/div/article/p[14]/a[1]"}, {"position_id": 1199, "txt": " because I used it a lot in the past and even contributed to ", "x": "/html/body/div/div/div/article/p[14]/a[1]"}, {"position_id": 1201, "txt": "the omit filter", "x": "/html/body/div/div/div/article/p[14]/a[2]"}, {"position_id": 1203, "txt": " many years ago.", "x": "/html/body/div/div/div/article/p[14]/a[2]"}]}, "tags": [], "label": "content", "web_segment_id": 29, "global_sentence_id": 73, "edu_l1_label": "IOS"}, {"txt": " So when I encountered this problem, Ansible came first into my mind for deploying Kubernetes configuration files and generating the PKI secrets plus certificates.", "language": "english", "position": {"atoms": [{"position_id": 1204, "txt": " So when I encountered this problem, Ansible came first into my mind for deploying Kubernetes configuration files and generating the PKI secrets plus certificates.", "x": "/html/body/div/div/div/article/p[14]/a[2]"}]}, "tags": [], "label": "content", "web_segment_id": 29, "global_sentence_id": 74, "edu_l1_label": "IOS"}, {"txt": " I don’t plan to cover too much detail, but here’s what I do.", "language": "english", "position": {"atoms": [{"position_id": 1205, "txt": " I don’t plan to cover too much detail, but here’s what I do.", "x": "/html/body/div/div/div/article/p[14]/a[2]"}]}, "tags": [], "label": "content", "web_segment_id": 29, "global_sentence_id": 75, "edu_l1_label": "IOS"}, {"txt": " I replace the Kubernetes services config in the NixOS configuration to make them read from a local file as the arguments like this:", "language": "english", "position": {"atoms": [{"position_id": 1206, "txt": " I replace the Kubernetes services config in the NixOS configuration to make them read from a local file as the arguments like this:", "x": "/html/body/div/div/div/article/p[14]/a[2]"}]}, "tags": [], "label": "content", "web_segment_id": 29, "global_sentence_id": 76, "edu_l1_label": "IOS"}, {"txt": "{config,pkgs,lib,...}:", "language": "english", "position": {"atoms": [{"position_id": 1208, "txt": "{", "x": "/html/body/div/div/div/article/div[2]/div/pre/code/span[1]"}, {"position_id": 1210, "txt": "config", "x": "/html/body/div/div/div/article/div[2]/div/pre/code/span[2]"}, {"position_id": 1212, "txt": ",", "x": "/html/body/div/div/div/article/div[2]/div/pre/code/span[3]"}, {"position_id": 1214, "txt": "pkgs", "x": "/html/body/div/div/div/article/div[2]/div/pre/code/span[4]"}, {"position_id": 1216, "txt": ",", "x": "/html/body/div/div/div/article/div[2]/div/pre/code/span[5]"}, {"position_id": 1218, "txt": "lib", "x": "/html/body/div/div/div/article/div[2]/div/pre/code/span[6]"}, {"position_id": 1220, "txt": ",", "x": "/html/body/div/div/div/article/div[2]/div/pre/code/span[7]"}, {"position_id": 1222, "txt": "...", "x": "/html/body/div/div/div/article/div[2]/div/pre/code/span[8]"}, {"position_id": 1224, "txt": "}:", "x": "/html/body/div/div/div/article/div[2]/div/pre/code/span[9]"}]}, "tags": [], "label": "code", "web_segment_id": 30, "global_sentence_id": 77, "edu_l1_label": "EDU_O"}, {"txt": "{systemd.services.kubelet={serviceConfig={Restart=mkForce\"always\";RestartSec=mkForce\"20s\";EnvironmentFile=\"/var/lib/k8s/kubelet.env\";ExecStart=mkForce''${pkgs.bash}/bin/bash -c \"${pkgs.kubernetes}/bin/kubelet $_KUBE_ARGS\"'';};};# ... other k8s services}", "language": "english", "position": {"atoms": [{"position_id": 1226, "txt": "{", "x": "/html/body/div/div/div/article/div[2]/div/pre/code/span[10]"}, {"position_id": 1228, "txt": "systemd", "x": "/html/body/div/div/div/article/div[2]/div/pre/code/span[11]"}, {"position_id": 1230, "txt": ".", "x": "/html/body/div/div/div/article/div[2]/div/pre/code/span[12]"}, {"position_id": 1232, "txt": "services", "x": "/html/body/div/div/div/article/div[2]/div/pre/code/span[13]"}, {"position_id": 1234, "txt": ".", "x": "/html/body/div/div/div/article/div[2]/div/pre/code/span[14]"}, {"position_id": 1236, "txt": "kubelet", "x": "/html/body/div/div/div/article/div[2]/div/pre/code/span[15]"}, {"position_id": 1238, "txt": "=", "x": "/html/body/div/div/div/article/div[2]/div/pre/code/span[16]"}, {"position_id": 1240, "txt": "{", "x": "/html/body/div/div/div/article/div[2]/div/pre/code/span[17]"}, {"position_id": 1242, "txt": "serviceConfig", "x": "/html/body/div/div/div/article/div[2]/div/pre/code/span[18]"}, {"position_id": 1244, "txt": "=", "x": "/html/body/div/div/div/article/div[2]/div/pre/code/span[19]"}, {"position_id": 1246, "txt": "{", "x": "/html/body/div/div/div/article/div[2]/div/pre/code/span[20]"}, {"position_id": 1248, "txt": "Restart", "x": "/html/body/div/div/div/article/div[2]/div/pre/code/span[21]"}, {"position_id": 1250, "txt": "=", "x": "/html/body/div/div/div/article/div[2]/div/pre/code/span[22]"}, {"position_id": 1252, "txt": "mkForce", "x": "/html/body/div/div/div/article/div[2]/div/pre/code/span[23]"}, {"position_id": 1254, "txt": "\"always\"", "x": "/html/body/div/div/div/article/div[2]/div/pre/code/span[24]"}, {"position_id": 1256, "txt": ";", "x": "/html/body/div/div/div/article/div[2]/div/pre/code/span[25]"}, {"position_id": 1258, "txt": "RestartSec", "x": "/html/body/div/div/div/article/div[2]/div/pre/code/span[26]"}, {"position_id": 1260, "txt": "=", "x": "/html/body/div/div/div/article/div[2]/div/pre/code/span[27]"}, {"position_id": 1262, "txt": "mkForce", "x": "/html/body/div/div/div/article/div[2]/div/pre/code/span[28]"}, {"position_id": 1264, "txt": "\"20s\"", "x": "/html/body/div/div/div/article/div[2]/div/pre/code/span[29]"}, {"position_id": 1266, "txt": ";", "x": "/html/body/div/div/div/article/div[2]/div/pre/code/span[30]"}, {"position_id": 1268, "txt": "EnvironmentFile", "x": "/html/body/div/div/div/article/div[2]/div/pre/code/span[31]"}, {"position_id": 1270, "txt": "=", "x": "/html/body/div/div/div/article/div[2]/div/pre/code/span[32]"}, {"position_id": 1272, "txt": "\"/var/lib/k8s/kubelet.env\"", "x": "/html/body/div/div/div/article/div[2]/div/pre/code/span[33]"}, {"position_id": 1274, "txt": ";", "x": "/html/body/div/div/div/article/div[2]/div/pre/code/span[34]"}, {"position_id": 1276, "txt": "ExecStart", "x": "/html/body/div/div/div/article/div[2]/div/pre/code/span[35]"}, {"position_id": 1278, "txt": "=", "x": "/html/body/div/div/div/article/div[2]/div/pre/code/span[36]"}, {"position_id": 1280, "txt": "mkForce", "x": "/html/body/div/div/div/article/div[2]/div/pre/code/span[37]"}, {"position_id": 1282, "txt": "''", "x": "/html/body/div/div/div/article/div[2]/div/pre/code/span[38]"}, {"position_id": 1284, "txt": "${", "x": "/html/body/div/div/div/article/div[2]/div/pre/code/span[39]"}, {"position_id": 1286, "txt": "pkgs", "x": "/html/body/div/div/div/article/div[2]/div/pre/code/span[40]"}, {"position_id": 1288, "txt": ".", "x": "/html/body/div/div/div/article/div[2]/div/pre/code/span[41]"}, {"position_id": 1290, "txt": "bash", "x": "/html/body/div/div/div/article/div[2]/div/pre/code/span[42]"}, {"position_id": 1292, "txt": "}", "x": "/html/body/div/div/div/article/div[2]/div/pre/code/span[43]"}, {"position_id": 1294, "txt": "/bin/bash -c \"", "x": "/html/body/div/div/div/article/div[2]/div/pre/code/span[44]"}, {"position_id": 1296, "txt": "${", "x": "/html/body/div/div/div/article/div[2]/div/pre/code/span[45]"}, {"position_id": 1298, "txt": "pkgs", "x": "/html/body/div/div/div/article/div[2]/div/pre/code/span[46]"}, {"position_id": 1300, "txt": ".", "x": "/html/body/div/div/div/article/div[2]/div/pre/code/span[47]"}, {"position_id": 1302, "txt": "kubernetes", "x": "/html/body/div/div/div/article/div[2]/div/pre/code/span[48]"}, {"position_id": 1304, "txt": "}", "x": "/html/body/div/div/div/article/div[2]/div/pre/code/span[49]"}, {"position_id": 1306, "txt": "/bin/kubelet $_KUBE_ARGS\"''", "x": "/html/body/div/div/div/article/div[2]/div/pre/code/span[50]"}, {"position_id": 1308, "txt": ";", "x": "/html/body/div/div/div/article/div[2]/div/pre/code/span[51]"}, {"position_id": 1310, "txt": "};", "x": "/html/body/div/div/div/article/div[2]/div/pre/code/span[52]"}, {"position_id": 1312, "txt": "};", "x": "/html/body/div/div/div/article/div[2]/div/pre/code/span[53]"}, {"position_id": 1314, "txt": "# ... other k8s services", "x": "/html/body/div/div/div/article/div[2]/div/pre/code/span[54]"}, {"position_id": 1316, "txt": "}", "x": "/html/body/div/div/div/article/div[2]/div/pre/code/span[55]"}]}, "tags": [], "label": "code", "web_segment_id": 30, "global_sentence_id": 78, "edu_l1_label": "EDU_O"}, {"txt": "Then, I have an Ansible playbook that will SSH into all machines and generate the corresponding arg and config files, along with PKI certificates and secret values.", "language": "english", "position": {"atoms": [{"position_id": 1318, "txt": "Then, I have an Ansible playbook that will SSH into all machines and generate the corresponding arg and config files, along with PKI certificates and secret values.", "x": "/html/body/div/div/div/article/p[15]"}]}, "tags": [], "label": "content", "web_segment_id": 31, "global_sentence_id": 79, "edu_l1_label": "IOS"}, {"txt": " Here’s an example of the Ansible Jinja2 template for generating the /var/lib/k8s/kubelet.env shown in the above Nix example:", "language": "english", "position": {"atoms": [{"position_id": 1319, "txt": " Here’s an example of the Ansible Jinja2 template for generating the ", "x": "/html/body/div/div/div/article/p[15]"}, {"position_id": 1321, "txt": "/var/lib/k8s/kubelet.env", "x": "/html/body/div/div/div/article/p[15]/code"}, {"position_id": 1323, "txt": " shown in the above Nix example:", "x": "/html/body/div/div/div/article/p[15]/code"}]}, "tags": [], "label": "content", "web_segment_id": 31, "global_sentence_id": 80, "edu_l1_label": "IOS"}, {"txt": "{{ansible_managed| comment}}_KUBE_ARGS={{([\"--root-dir=/var/lib/k8s/kubelet\",\"--config=/var/lib/k8s/kubelet/config.yaml\",\"--kubeconfig=/var/lib/k8s/pki/kubelet.conf\",\"--image-credential-provider-config=/var/lib/k8s/kubelet/image-credential-provider.yaml\",\"--image-credential-provider-bin-dir=/run/current-system/sw/bin\",\"--runtime-cgroups=/system.slice/containerd.service\",\"--node-ip={}\".format(nebula1_node_ip| ansible.utils.ipaddr('address')),\"-v\",(kubelet_log_level| default(1)| quote),]+([\"--node-labels={}\".format(','.join(node_labels))]ifnode_labelsisdefinedelse[]))| join(\" \")| quote}}", "language": "english", "position": {"atoms": [{"position_id": 1325, "txt": "{{", "x": "/html/body/div/div/div/article/div[3]/div/pre/code/span[1]"}, {"position_id": 1327, "txt": "ansible_managed", "x": "/html/body/div/div/div/article/div[3]/div/pre/code/span[2]"}, {"position_id": 1329, "txt": "| ", "x": "/html/body/div/div/div/article/div[3]/div/pre/code/span[3]"}, {"position_id": 1331, "txt": "comment", "x": "/html/body/div/div/div/article/div[3]/div/pre/code/span[4]"}, {"position_id": 1333, "txt": "}}", "x": "/html/body/div/div/div/article/div[3]/div/pre/code/span[5]"}, {"position_id": 1335, "txt": "\n\n_KUBE_ARGS=", "x": "/html/body/div/div/div/article/div[3]/div/pre/code/span[5]"}, {"position_id": 1337, "txt": "{{", "x": "/html/body/div/div/div/article/div[3]/div/pre/code/span[6]"}, {"position_id": 1339, "txt": "(", "x": "/html/body/div/div/div/article/div[3]/div/pre/code/span[7]"}, {"position_id": 1341, "txt": "[", "x": "/html/body/div/div/div/article/div[3]/div/pre/code/span[8]"}, {"position_id": 1343, "txt": "\"--root-dir=/var/lib/k8s/kubelet\"", "x": "/html/body/div/div/div/article/div[3]/div/pre/code/span[9]"}, {"position_id": 1345, "txt": ",", "x": "/html/body/div/div/div/article/div[3]/div/pre/code/span[10]"}, {"position_id": 1347, "txt": "\"--config=/var/lib/k8s/kubelet/config.yaml\"", "x": "/html/body/div/div/div/article/div[3]/div/pre/code/span[11]"}, {"position_id": 1349, "txt": ",", "x": "/html/body/div/div/div/article/div[3]/div/pre/code/span[12]"}, {"position_id": 1351, "txt": "\"--kubeconfig=/var/lib/k8s/pki/kubelet.conf\"", "x": "/html/body/div/div/div/article/div[3]/div/pre/code/span[13]"}, {"position_id": 1353, "txt": ",", "x": "/html/body/div/div/div/article/div[3]/div/pre/code/span[14]"}, {"position_id": 1355, "txt": "\"--image-credential-provider-config=/var/lib/k8s/kubelet/image-credential-provider.yaml\"", "x": "/html/body/div/div/div/article/div[3]/div/pre/code/span[15]"}, {"position_id": 1357, "txt": ",", "x": "/html/body/div/div/div/article/div[3]/div/pre/code/span[16]"}, {"position_id": 1359, "txt": "\"--image-credential-provider-bin-dir=/run/current-system/sw/bin\"", "x": "/html/body/div/div/div/article/div[3]/div/pre/code/span[17]"}, {"position_id": 1361, "txt": ",", "x": "/html/body/div/div/div/article/div[3]/div/pre/code/span[18]"}, {"position_id": 1363, "txt": "\"--runtime-cgroups=/system.slice/containerd.service\"", "x": "/html/body/div/div/div/article/div[3]/div/pre/code/span[19]"}, {"position_id": 1365, "txt": ",", "x": "/html/body/div/div/div/article/div[3]/div/pre/code/span[20]"}, {"position_id": 1367, "txt": "\"--node-ip={}\"", "x": "/html/body/div/div/div/article/div[3]/div/pre/code/span[21]"}, {"position_id": 1369, "txt": ".", "x": "/html/body/div/div/div/article/div[3]/div/pre/code/span[22]"}, {"position_id": 1371, "txt": "format", "x": "/html/body/div/div/div/article/div[3]/div/pre/code/span[23]"}, {"position_id": 1373, "txt": "(", "x": "/html/body/div/div/div/article/div[3]/div/pre/code/span[24]"}, {"position_id": 1375, "txt": "nebula1_node_ip", "x": "/html/body/div/div/div/article/div[3]/div/pre/code/span[25]"}, {"position_id": 1377, "txt": "| ", "x": "/html/body/div/div/div/article/div[3]/div/pre/code/span[26]"}, {"position_id": 1379, "txt": "ansible", "x": "/html/body/div/div/div/article/div[3]/div/pre/code/span[27]"}, {"position_id": 1381, "txt": ".", "x": "/html/body/div/div/div/article/div[3]/div/pre/code/span[28]"}, {"position_id": 1383, "txt": "utils.ipaddr", "x": "/html/body/div/div/div/article/div[3]/div/pre/code/span[29]"}, {"position_id": 1385, "txt": "(", "x": "/html/body/div/div/div/article/div[3]/div/pre/code/span[30]"}, {"position_id": 1387, "txt": "'address'", "x": "/html/body/div/div/div/article/div[3]/div/pre/code/span[31]"}, {"position_id": 1389, "txt": ")),", "x": "/html/body/div/div/div/article/div[3]/div/pre/code/span[32]"}, {"position_id": 1391, "txt": "\"-v\"", "x": "/html/body/div/div/div/article/div[3]/div/pre/code/span[33]"}, {"position_id": 1393, "txt": ",", "x": "/html/body/div/div/div/article/div[3]/div/pre/code/span[34]"}, {"position_id": 1395, "txt": "(", "x": "/html/body/div/div/div/article/div[3]/div/pre/code/span[35]"}, {"position_id": 1397, "txt": "kubelet_log_level", "x": "/html/body/div/div/div/article/div[3]/div/pre/code/span[36]"}, {"position_id": 1399, "txt": "| ", "x": "/html/body/div/div/div/article/div[3]/div/pre/code/span[37]"}, {"position_id": 1401, "txt": "default", "x": "/html/body/div/div/div/article/div[3]/div/pre/code/span[38]"}, {"position_id": 1403, "txt": "(", "x": "/html/body/div/div/div/article/div[3]/div/pre/code/span[39]"}, {"position_id": 1405, "txt": "1", "x": "/html/body/div/div/div/article/div[3]/div/pre/code/span[40]"}, {"position_id": 1407, "txt": ")", "x": "/html/body/div/div/div/article/div[3]/div/pre/code/span[41]"}, {"position_id": 1409, "txt": "| ", "x": "/html/body/div/div/div/article/div[3]/div/pre/code/span[42]"}, {"position_id": 1411, "txt": "quote", "x": "/html/body/div/div/div/article/div[3]/div/pre/code/span[43]"}, {"position_id": 1413, "txt": "),", "x": "/html/body/div/div/div/article/div[3]/div/pre/code/span[44]"}, {"position_id": 1415, "txt": "]", "x": "/html/body/div/div/div/article/div[3]/div/pre/code/span[45]"}, {"position_id": 1417, "txt": "+", "x": "/html/body/div/div/div/article/div[3]/div/pre/code/span[46]"}, {"position_id": 1419, "txt": "([", "x": "/html/body/div/div/div/article/div[3]/div/pre/code/span[47]"}, {"position_id": 1421, "txt": "\"--node-labels={}\"", "x": "/html/body/div/div/div/article/div[3]/div/pre/code/span[48]"}, {"position_id": 1423, "txt": ".", "x": "/html/body/div/div/div/article/div[3]/div/pre/code/span[49]"}, {"position_id": 1425, "txt": "format", "x": "/html/body/div/div/div/article/div[3]/div/pre/code/span[50]"}, {"position_id": 1427, "txt": "(", "x": "/html/body/div/div/div/article/div[3]/div/pre/code/span[51]"}, {"position_id": 1429, "txt": "','", "x": "/html/body/div/div/div/article/div[3]/div/pre/code/span[52]"}, {"position_id": 1431, "txt": ".", "x": "/html/body/div/div/div/article/div[3]/div/pre/code/span[53]"}, {"position_id": 1433, "txt": "join", "x": "/html/body/div/div/div/article/div[3]/div/pre/code/span[54]"}, {"position_id": 1435, "txt": "(", "x": "/html/body/div/div/div/article/div[3]/div/pre/code/span[55]"}, {"position_id": 1437, "txt": "node_labels", "x": "/html/body/div/div/div/article/div[3]/div/pre/code/span[56]"}, {"position_id": 1439, "txt": "))]", "x": "/html/body/div/div/div/article/div[3]/div/pre/code/span[57]"}, {"position_id": 1441, "txt": "if", "x": "/html/body/div/div/div/article/div[3]/div/pre/code/span[58]"}, {"position_id": 1443, "txt": "node_labels", "x": "/html/body/div/div/div/article/div[3]/div/pre/code/span[59]"}, {"position_id": 1445, "txt": "is", "x": "/html/body/div/div/div/article/div[3]/div/pre/code/span[60]"}, {"position_id": 1447, "txt": "defined", "x": "/html/body/div/div/div/article/div[3]/div/pre/code/span[61]"}, {"position_id": 1449, "txt": "else", "x": "/html/body/div/div/div/article/div[3]/div/pre/code/span[62]"}, {"position_id": 1451, "txt": "[])", "x": "/html/body/div/div/div/article/div[3]/div/pre/code/span[63]"}, {"position_id": 1453, "txt": ")", "x": "/html/body/div/div/div/article/div[3]/div/pre/code/span[64]"}, {"position_id": 1455, "txt": "| ", "x": "/html/body/div/div/div/article/div[3]/div/pre/code/span[65]"}, {"position_id": 1457, "txt": "join", "x": "/html/body/div/div/div/article/div[3]/div/pre/code/span[66]"}, {"position_id": 1459, "txt": "(", "x": "/html/body/div/div/div/article/div[3]/div/pre/code/span[67]"}, {"position_id": 1461, "txt": "\" \"", "x": "/html/body/div/div/div/article/div[3]/div/pre/code/span[68]"}, {"position_id": 1463, "txt": ")", "x": "/html/body/div/div/div/article/div[3]/div/pre/code/span[69]"}, {"position_id": 1465, "txt": "| ", "x": "/html/body/div/div/div/article/div[3]/div/pre/code/span[70]"}, {"position_id": 1467, "txt": "quote", "x": "/html/body/div/div/div/article/div[3]/div/pre/code/span[71]"}, {"position_id": 1469, "txt": "}}", "x": "/html/body/div/div/div/article/div[3]/div/pre/code/span[72]"}]}, "tags": [], "label": "code", "web_segment_id": 32, "global_sentence_id": 81, "edu_l1_label": "EDU_O"}, {"txt": "You may ask where you put the secret value.", "language": "english", "position": {"atoms": [{"position_id": 1471, "txt": "You may ask where you put the secret value.", "x": "/html/body/div/div/div/article/p[16]"}]}, "tags": [], "label": "content", "web_segment_id": 33, "global_sentence_id": 82, "edu_l1_label": "IOS"}, {"txt": " I use Sops and its Ansible plugin.", "language": "english", "position": {"atoms": [{"position_id": 1472, "txt": " I use ", "x": "/html/body/div/div/div/article/p[16]"}, {"position_id": 1474, "txt": "Sops", "x": "/html/body/div/div/div/article/p[16]/a[1]"}, {"position_id": 1476, "txt": " and its ", "x": "/html/body/div/div/div/article/p[16]/a[1]"}, {"position_id": 1478, "txt": "Ansible plugin", "x": "/html/body/div/div/div/article/p[16]/a[2]"}, {"position_id": 1480, "txt": ".", "x": "/html/body/div/div/div/article/p[16]/a[2]"}]}, "tags": [], "label": "content", "web_segment_id": 33, "global_sentence_id": 83, "edu_l1_label": "IOS"}, {"txt": " All the values are encrypted using a GPG key.", "language": "english", "position": {"atoms": [{"position_id": 1481, "txt": " All the values are encrypted using a GPG key.", "x": "/html/body/div/div/div/article/p[16]/a[2]"}]}, "tags": [], "label": "content", "web_segment_id": 33, "global_sentence_id": 84, "edu_l1_label": "IOS"}, {"txt": "-name:", "language": "english", "position": {"atoms": [{"position_id": 1483, "txt": "-", "x": "/html/body/div/div/div/article/div[4]/div/pre/code/span[1]"}, {"position_id": 1485, "txt": "name", "x": "/html/body/div/div/div/article/div[4]/div/pre/code/span[2]"}, {"position_id": 1487, "txt": ":", "x": "/html/body/div/div/div/article/div[4]/div/pre/code/span[3]"}]}, "tags": [], "label": "content", "web_segment_id": 34, "global_sentence_id": 85, "edu_l1_label": "EDU_O"}, {"txt": "Deploy k8s clusterhosts:", "language": "english", "position": {"atoms": [{"position_id": 1489, "txt": "Deploy k8s cluster", "x": "/html/body/div/div/div/article/div[4]/div/pre/code/span[4]"}, {"position_id": 1491, "txt": "hosts", "x": "/html/body/div/div/div/article/div[4]/div/pre/code/span[5]"}, {"position_id": 1493, "txt": ":", "x": "/html/body/div/div/div/article/div[4]/div/pre/code/span[6]"}]}, "tags": [], "label": "code", "web_segment_id": 34, "global_sentence_id": 86, "edu_l1_label": "EDU_O"}, {"txt": "allpre_tasks:", "language": "english", "position": {"atoms": [{"position_id": 1495, "txt": "all", "x": "/html/body/div/div/div/article/div[4]/div/pre/code/span[7]"}, {"position_id": 1497, "txt": "pre_tasks", "x": "/html/body/div/div/div/article/div[4]/div/pre/code/span[8]"}, {"position_id": 1499, "txt": ":", "x": "/html/body/div/div/div/article/div[4]/div/pre/code/span[9]"}]}, "tags": [], "label": "code", "web_segment_id": 34, "global_sentence_id": 87, "edu_l1_label": "EDU_O"}, {"txt": "-import_tasks:", "language": "english", "position": {"atoms": [{"position_id": 1501, "txt": "-", "x": "/html/body/div/div/div/article/div[4]/div/pre/code/span[10]"}, {"position_id": 1503, "txt": "import_tasks", "x": "/html/body/div/div/div/article/div[4]/div/pre/code/span[11]"}, {"position_id": 1505, "txt": ":", "x": "/html/body/div/div/div/article/div[4]/div/pre/code/span[12]"}]}, "tags": [], "label": "code", "web_segment_id": 34, "global_sentence_id": 88, "edu_l1_label": "EDU_O"}, {"txt": "tasks/sops.yamltags:", "language": "english", "position": {"atoms": [{"position_id": 1507, "txt": "tasks/sops.yaml", "x": "/html/body/div/div/div/article/div[4]/div/pre/code/span[13]"}, {"position_id": 1509, "txt": "tags", "x": "/html/body/div/div/div/article/div[4]/div/pre/code/span[14]"}, {"position_id": 1511, "txt": ":", "x": "/html/body/div/div/div/article/div[4]/div/pre/code/span[15]"}]}, "tags": [], "label": "code", "web_segment_id": 34, "global_sentence_id": 89, "edu_l1_label": "EDU_O"}, {"txt": "[\"sops\",\"always\"]tasks:", "language": "english", "position": {"atoms": [{"position_id": 1513, "txt": "[", "x": "/html/body/div/div/div/article/div[4]/div/pre/code/span[16]"}, {"position_id": 1515, "txt": "\"", "x": "/html/body/div/div/div/article/div[4]/div/pre/code/span[17]"}, {"position_id": 1517, "txt": "sops\"", "x": "/html/body/div/div/div/article/div[4]/div/pre/code/span[18]"}, {"position_id": 1519, "txt": ",", "x": "/html/body/div/div/div/article/div[4]/div/pre/code/span[19]"}, {"position_id": 1521, "txt": "\"", "x": "/html/body/div/div/div/article/div[4]/div/pre/code/span[20]"}, {"position_id": 1523, "txt": "always\"", "x": "/html/body/div/div/div/article/div[4]/div/pre/code/span[21]"}, {"position_id": 1525, "txt": "]", "x": "/html/body/div/div/div/article/div[4]/div/pre/code/span[22]"}, {"position_id": 1527, "txt": "tasks", "x": "/html/body/div/div/div/article/div[4]/div/pre/code/span[23]"}, {"position_id": 1529, "txt": ":", "x": "/html/body/div/div/div/article/div[4]/div/pre/code/span[24]"}]}, "tags": [], "label": "code", "web_segment_id": 34, "global_sentence_id": 90, "edu_l1_label": "EDU_O"}, {"txt": "-name:", "language": "english", "position": {"atoms": [{"position_id": 1531, "txt": "-", "x": "/html/body/div/div/div/article/div[4]/div/pre/code/span[25]"}, {"position_id": 1533, "txt": "name", "x": "/html/body/div/div/div/article/div[4]/div/pre/code/span[26]"}, {"position_id": 1535, "txt": ":", "x": "/html/body/div/div/div/article/div[4]/div/pre/code/span[27]"}]}, "tags": [], "label": "code", "web_segment_id": 34, "global_sentence_id": 91, "edu_l1_label": "EDU_O"}, {"txt": "Do something with the secret valueansible.builtin.debug:", "language": "english", "position": {"atoms": [{"position_id": 1537, "txt": "Do something with the secret value", "x": "/html/body/div/div/div/article/div[4]/div/pre/code/span[28]"}, {"position_id": 1539, "txt": "ansible.builtin.debug", "x": "/html/body/div/div/div/article/div[4]/div/pre/code/span[29]"}, {"position_id": 1541, "txt": ":", "x": "/html/body/div/div/div/article/div[4]/div/pre/code/span[30]"}]}, "tags": [], "label": "code", "web_segment_id": 34, "global_sentence_id": 92, "edu_l1_label": "EDU_O"}, {"txt": "msg:\"", "language": "english", "position": {"atoms": [{"position_id": 1543, "txt": "msg", "x": "/html/body/div/div/div/article/div[4]/div/pre/code/span[31]"}, {"position_id": 1545, "txt": ":", "x": "/html/body/div/div/div/article/div[4]/div/pre/code/span[32]"}, {"position_id": 1547, "txt": "\"", "x": "/html/body/div/div/div/article/div[4]/div/pre/code/span[33]"}]}, "tags": [], "label": "code", "web_segment_id": 34, "global_sentence_id": 93, "edu_l1_label": "EDU_O"}, {"txt": "{{secrets.my-value}}\"# ...", "language": "english", "position": {"atoms": [{"position_id": 1549, "txt": "{{", "x": "/html/body/div/div/div/article/div[4]/div/pre/code/span[34]"}, {"position_id": 1551, "txt": "secrets.my-value", "x": "/html/body/div/div/div/article/div[4]/div/pre/code/span[36]"}, {"position_id": 1553, "txt": "}}\"", "x": "/html/body/div/div/div/article/div[4]/div/pre/code/span[38]"}, {"position_id": 1555, "txt": "# ...", "x": "/html/body/div/div/div/article/div[4]/div/pre/code/span[39]"}]}, "tags": [], "label": "code", "web_segment_id": 34, "global_sentence_id": 94, "edu_l1_label": "EDU_O"}, {"txt": "By combining NixOS, Ansible, and Sops, I can easily deploy to as many bare metal machines as I want as a Kubernetes cluster.", "language": "english", "position": {"atoms": [{"position_id": 1557, "txt": "By combining NixOS, Ansible, and Sops, I can easily deploy to as many bare metal machines as I want as a Kubernetes cluster.", "x": "/html/body/div/div/div/article/p[17]"}]}, "tags": [], "label": "content", "web_segment_id": 35, "global_sentence_id": 95, "edu_l1_label": "IOS"}, {"txt": " I even built a NixOS live CD environment for a USB thumb drive to help speed up the bootstrapping process.", "language": "english", "position": {"atoms": [{"position_id": 1558, "txt": " I even built ", "x": "/html/body/div/div/div/article/p[17]"}, {"position_id": 1560, "txt": "a NixOS live CD environment for a USB thumb", "x": "/html/body/div/div/div/article/p[17]/a"}, {"position_id": 1562, "txt": " drive to help speed up the bootstrapping process.", "x": "/html/body/div/div/div/article/p[17]/a"}]}, "tags": [], "label": "content", "web_segment_id": 35, "global_sentence_id": 96, "edu_l1_label": "IOS"}, {"txt": " But this article already has too much content, so I will leave it for text time.", "language": "english", "position": {"atoms": [{"position_id": 1563, "txt": " But this article already has too much content, so I will leave it for text time.", "x": "/html/body/div/div/div/article/p[17]/a"}]}, "tags": [], "label": "content", "web_segment_id": 35, "global_sentence_id": 97, "edu_l1_label": "IOS"}, {"txt": "Kubernetes with Nvidia GPU CDI plugin explained", "language": "english", "position": {"atoms": [{"position_id": 1565, "txt": "Kubernetes with Nvidia GPU CDI plugin explained", "x": "/html/body/div/div/div/article/h2[3]"}]}, "tags": ["h2"], "label": "title1", "web_segment_id": 36, "global_sentence_id": 98, "edu_l1_label": "BOS"}, {"txt": "Running a pod on Kubernetes with Nvidia GPU exposes sounds straightforward.", "language": "english", "position": {"atoms": [{"position_id": 1567, "txt": "Running a pod on Kubernetes with Nvidia GPU exposes sounds straightforward.", "x": "/html/body/div/div/div/article/p[18]"}]}, "tags": [], "label": "content", "web_segment_id": 37, "global_sentence_id": 99, "edu_l1_label": "IOS"}, {"txt": " How hard could it be?", "language": "english", "position": {"atoms": [{"position_id": 1568, "txt": " How hard could it be?", "x": "/html/body/div/div/div/article/p[18]"}]}, "tags": [], "label": "content", "web_segment_id": 37, "global_sentence_id": 100, "edu_l1_label": "IOS"}, {"txt": " But when you look closer, it immediately gives you a headache.", "language": "english", "position": {"atoms": [{"position_id": 1569, "txt": " But when you look closer, it immediately gives you a headache.", "x": "/html/body/div/div/div/article/p[18]"}]}, "tags": [], "label": "content", "web_segment_id": 37, "global_sentence_id": 101, "edu_l1_label": "IOS"}, {"txt": " Just the sheer number of terms may overwhelm some people.", "language": "english", "position": {"atoms": [{"position_id": 1570, "txt": " Just the sheer number of terms may overwhelm some people.", "x": "/html/body/div/div/div/article/p[18]"}]}, "tags": [], "label": "content", "web_segment_id": 37, "global_sentence_id": 102, "edu_l1_label": "IOS"}, {"txt": " Be it CRI, CDI, nvidia-container-toolkit, libnvidia-container, and all the confusing terms at first glance.", "language": "english", "position": {"atoms": [{"position_id": 1571, "txt": " Be it CRI, CDI, nvidia-container-toolkit, libnvidia-container, and all the confusing terms at first glance.", "x": "/html/body/div/div/div/article/p[18]"}]}, "tags": [], "label": "content", "web_segment_id": 37, "global_sentence_id": 103, "edu_l1_label": "IOS"}, {"txt": "Although I have plenty of experience in Kubernetes and containertechnologies, it also took me a while to fully understand those terms and how they are interconnected.", "language": "english", "position": {"atoms": [{"position_id": 1573, "txt": "Although I have plenty of experience in Kubernetes and ", "x": "/html/body/div/div/div/article/p[19]"}, {"position_id": 1575, "txt": "container", "x": "/html/body/div/div/div/article/p[19]/a[1]"}, {"position_id": 1577, "txt": "technologies", "x": "/html/body/div/div/div/article/p[19]/a[2]"}, {"position_id": 1579, "txt": ", it also took me a while to fully understand those terms and how they are interconnected.", "x": "/html/body/div/div/div/article/p[19]/a[2]"}]}, "tags": [], "label": "content", "web_segment_id": 38, "global_sentence_id": 104, "edu_l1_label": "IOS"}, {"txt": " Let’s first see the big picture of how everything works together, and then we will explore the details of how each component works together.", "language": "english", "position": {"atoms": [{"position_id": 1580, "txt": " Let’s first see the big picture of how everything works together, and then we will explore the details of how each component works together.", "x": "/html/body/div/div/div/article/p[19]/a[2]"}]}, "tags": [], "label": "content", "web_segment_id": 38, "global_sentence_id": 105, "edu_l1_label": "IOS"}, {"txt": "<img img", "language": "english", "position": {"atoms": [{"position_id": 349, "txt": "<img img", "x": "/html/body/div/div/div/article/figure[6]/img"}]}, "tags": ["img"], "label": "figure", "web_segment_id": 2, "global_sentence_id": 106, "edu_l1_label": "EDU_O"}, {"txt": "Schematic overview of the Nvidia GPU integration architecture within Kubernetes on NixOS, showcasing the interplay between Container Runtime Interface (CRI), Container Device Interface (CDI), nvidia-container-toolkit, and the k8s-device-plugin.", "language": "english", "position": {"atoms": [{"position_id": 1582, "txt": "Schematic overview of the Nvidia GPU integration architecture within Kubernetes on NixOS, showcasing the interplay between Container Runtime Interface (CRI), Container Device Interface (CDI), nvidia-container-toolkit, and the k8s-device-plugin.", "x": "/html/body/div/div/div/article/figure[6]/figcaption/p"}]}, "tags": ["figcaption", "img"], "label": "figure_title", "web_segment_id": 39, "global_sentence_id": 107, "edu_l1_label": "EDU_O"}, {"txt": "The official document from Nvidia about the architecture will also help you understand.", "language": "english", "position": {"atoms": [{"position_id": 1584, "txt": "The official ", "x": "/html/body/div/div/div/article/p[20]"}, {"position_id": 1586, "txt": "document from Nvidia about the architecture", "x": "/html/body/div/div/div/article/p[20]/a"}, {"position_id": 1588, "txt": " will also help you understand.", "x": "/html/body/div/div/div/article/p[20]/a"}]}, "tags": [], "label": "content", "web_segment_id": 40, "global_sentence_id": 108, "edu_l1_label": "IOS"}, {"txt": "Container Runtime Interface", "language": "english", "position": {"atoms": [{"position_id": 1590, "txt": "Container Runtime Interface", "x": "/html/body/div/div/div/article/h3[1]"}]}, "tags": ["h3"], "label": "title2", "web_segment_id": 41, "global_sentence_id": 109, "edu_l1_label": "IOS"}, {"txt": "As you are familiar with Kubernetes, there are so many CXI terms, such as CNI (Container Network Interface) or CSI (Container Storage Interface); they are all well-defined interfaces that open up the implementation details of fundamental Kubernetes functionalities to third parties.", "language": "english", "position": {"atoms": [{"position_id": 1592, "txt": "As you are familiar with Kubernetes, there are so many CXI terms, such as ", "x": "/html/body/div/div/div/article/p[21]"}, {"position_id": 1594, "txt": "CNI (Container Network Interface)", "x": "/html/body/div/div/div/article/p[21]/a[1]"}, {"position_id": 1596, "txt": " or ", "x": "/html/body/div/div/div/article/p[21]/a[1]"}, {"position_id": 1598, "txt": "CSI (Container Storage Interface)", "x": "/html/body/div/div/div/article/p[21]/a[2]"}, {"position_id": 1600, "txt": "; they are all well-defined interfaces that open up the implementation details of fundamental Kubernetes functionalities to third parties.", "x": "/html/body/div/div/div/article/p[21]/a[2]"}]}, "tags": [], "label": "content", "web_segment_id": 42, "global_sentence_id": 110, "edu_l1_label": "IOS"}, {"txt": " For instance, CNI is for networking, and CSI is for data storage.", "language": "english", "position": {"atoms": [{"position_id": 1601, "txt": " For instance, CNI is for networking, and CSI is for data storage.", "x": "/html/body/div/div/div/article/p[21]/a[2]"}]}, "tags": [], "label": "content", "web_segment_id": 42, "global_sentence_id": 111, "edu_l1_label": "IOS"}, {"txt": "Likewise, CRI stands for Container Runtime Interface, the plugin interface for adopting different container management systems.", "language": "english", "position": {"atoms": [{"position_id": 1603, "txt": "Likewise, CRI stands for ", "x": "/html/body/div/div/div/article/p[22]"}, {"position_id": 1605, "txt": "Container Runtime Interface", "x": "/html/body/div/div/div/article/p[22]/a[1]"}, {"position_id": 1607, "txt": ", the plugin interface for adopting different container management systems.", "x": "/html/body/div/div/div/article/p[22]/a[1]"}]}, "tags": [], "label": "content", "web_segment_id": 43, "global_sentence_id": 112, "edu_l1_label": "IOS"}, {"txt": " The most well-known ones are Containerd and CRI-O.", "language": "english", "position": {"atoms": [{"position_id": 1608, "txt": " The most well-known ones are ", "x": "/html/body/div/div/div/article/p[22]/a[1]"}, {"position_id": 1610, "txt": "Containerd", "x": "/html/body/div/div/div/article/p[22]/a[2]"}, {"position_id": 1612, "txt": " and ", "x": "/html/body/div/div/div/article/p[22]/a[2]"}, {"position_id": 1614, "txt": "CRI-O", "x": "/html/body/div/div/div/article/p[22]/a[3]"}, {"position_id": 1616, "txt": ".", "x": "/html/body/div/div/div/article/p[22]/a[3]"}]}, "tags": [], "label": "content", "web_segment_id": 43, "global_sentence_id": 113, "edu_l1_label": "IOS"}, {"txt": " With NixOS, you can change its settings like this to make it support Nvidia runtime:", "language": "english", "position": {"atoms": [{"position_id": 1617, "txt": " With NixOS, you can change its settings like this to make it support Nvidia runtime:", "x": "/html/body/div/div/div/article/p[22]/a[3]"}]}, "tags": [], "label": "content", "web_segment_id": 43, "global_sentence_id": 114, "edu_l1_label": "IOS"}, {"txt": "virtualisation.containerd={settings={plugins={\"io.containerd.grpc.v1.cri\"={enable_cdi=true;cdi_spec_dirs=[\"/etc/cdi\"\"/var/run/cdi\"];containerd={default_runtime_name=\"nvidia\";runtimes={nvidia={privileged_without_host_devices=false;runtime_type=\"io.containerd.runc.v2\";options={BinaryName=\"${pkgs.nvidia-container-toolkit}/bin/nvidia-container-runtime\";};};};};};};};};", "language": "english", "position": {"atoms": [{"position_id": 1619, "txt": "virtualisation", "x": "/html/body/div/div/div/article/div[5]/div/pre/code/span[1]"}, {"position_id": 1621, "txt": ".", "x": "/html/body/div/div/div/article/div[5]/div/pre/code/span[2]"}, {"position_id": 1623, "txt": "containerd", "x": "/html/body/div/div/div/article/div[5]/div/pre/code/span[3]"}, {"position_id": 1625, "txt": "=", "x": "/html/body/div/div/div/article/div[5]/div/pre/code/span[4]"}, {"position_id": 1627, "txt": "{", "x": "/html/body/div/div/div/article/div[5]/div/pre/code/span[5]"}, {"position_id": 1629, "txt": "settings", "x": "/html/body/div/div/div/article/div[5]/div/pre/code/span[6]"}, {"position_id": 1631, "txt": "=", "x": "/html/body/div/div/div/article/div[5]/div/pre/code/span[7]"}, {"position_id": 1633, "txt": "{", "x": "/html/body/div/div/div/article/div[5]/div/pre/code/span[8]"}, {"position_id": 1635, "txt": "plugins", "x": "/html/body/div/div/div/article/div[5]/div/pre/code/span[9]"}, {"position_id": 1637, "txt": "=", "x": "/html/body/div/div/div/article/div[5]/div/pre/code/span[10]"}, {"position_id": 1639, "txt": "{", "x": "/html/body/div/div/div/article/div[5]/div/pre/code/span[11]"}, {"position_id": 1641, "txt": "\"io.containerd.grpc.v1.cri\"", "x": "/html/body/div/div/div/article/div[5]/div/pre/code/span[12]"}, {"position_id": 1643, "txt": "=", "x": "/html/body/div/div/div/article/div[5]/div/pre/code/span[13]"}, {"position_id": 1645, "txt": "{", "x": "/html/body/div/div/div/article/div[5]/div/pre/code/span[14]"}, {"position_id": 1647, "txt": "enable_cdi", "x": "/html/body/div/div/div/article/div[5]/div/pre/code/span[15]"}, {"position_id": 1649, "txt": "=", "x": "/html/body/div/div/div/article/div[5]/div/pre/code/span[16]"}, {"position_id": 1651, "txt": "true", "x": "/html/body/div/div/div/article/div[5]/div/pre/code/span[17]"}, {"position_id": 1653, "txt": ";", "x": "/html/body/div/div/div/article/div[5]/div/pre/code/span[18]"}, {"position_id": 1655, "txt": "cdi_spec_dirs", "x": "/html/body/div/div/div/article/div[5]/div/pre/code/span[19]"}, {"position_id": 1657, "txt": "=", "x": "/html/body/div/div/div/article/div[5]/div/pre/code/span[20]"}, {"position_id": 1659, "txt": "[", "x": "/html/body/div/div/div/article/div[5]/div/pre/code/span[21]"}, {"position_id": 1661, "txt": "\"/etc/cdi\"", "x": "/html/body/div/div/div/article/div[5]/div/pre/code/span[22]"}, {"position_id": 1663, "txt": "\"/var/run/cdi\"", "x": "/html/body/div/div/div/article/div[5]/div/pre/code/span[23]"}, {"position_id": 1665, "txt": "];", "x": "/html/body/div/div/div/article/div[5]/div/pre/code/span[24]"}, {"position_id": 1667, "txt": "containerd", "x": "/html/body/div/div/div/article/div[5]/div/pre/code/span[25]"}, {"position_id": 1669, "txt": "=", "x": "/html/body/div/div/div/article/div[5]/div/pre/code/span[26]"}, {"position_id": 1671, "txt": "{", "x": "/html/body/div/div/div/article/div[5]/div/pre/code/span[27]"}, {"position_id": 1673, "txt": "default_runtime_name", "x": "/html/body/div/div/div/article/div[5]/div/pre/code/span[28]"}, {"position_id": 1675, "txt": "=", "x": "/html/body/div/div/div/article/div[5]/div/pre/code/span[29]"}, {"position_id": 1677, "txt": "\"nvidia\"", "x": "/html/body/div/div/div/article/div[5]/div/pre/code/span[30]"}, {"position_id": 1679, "txt": ";", "x": "/html/body/div/div/div/article/div[5]/div/pre/code/span[31]"}, {"position_id": 1681, "txt": "runtimes", "x": "/html/body/div/div/div/article/div[5]/div/pre/code/span[32]"}, {"position_id": 1683, "txt": "=", "x": "/html/body/div/div/div/article/div[5]/div/pre/code/span[33]"}, {"position_id": 1685, "txt": "{", "x": "/html/body/div/div/div/article/div[5]/div/pre/code/span[34]"}, {"position_id": 1687, "txt": "nvidia", "x": "/html/body/div/div/div/article/div[5]/div/pre/code/span[35]"}, {"position_id": 1689, "txt": "=", "x": "/html/body/div/div/div/article/div[5]/div/pre/code/span[36]"}, {"position_id": 1691, "txt": "{", "x": "/html/body/div/div/div/article/div[5]/div/pre/code/span[37]"}, {"position_id": 1693, "txt": "privileged_without_host_devices", "x": "/html/body/div/div/div/article/div[5]/div/pre/code/span[38]"}, {"position_id": 1695, "txt": "=", "x": "/html/body/div/div/div/article/div[5]/div/pre/code/span[39]"}, {"position_id": 1697, "txt": "false", "x": "/html/body/div/div/div/article/div[5]/div/pre/code/span[40]"}, {"position_id": 1699, "txt": ";", "x": "/html/body/div/div/div/article/div[5]/div/pre/code/span[41]"}, {"position_id": 1701, "txt": "runtime_type", "x": "/html/body/div/div/div/article/div[5]/div/pre/code/span[42]"}, {"position_id": 1703, "txt": "=", "x": "/html/body/div/div/div/article/div[5]/div/pre/code/span[43]"}, {"position_id": 1705, "txt": "\"io.containerd.runc.v2\"", "x": "/html/body/div/div/div/article/div[5]/div/pre/code/span[44]"}, {"position_id": 1707, "txt": ";", "x": "/html/body/div/div/div/article/div[5]/div/pre/code/span[45]"}, {"position_id": 1709, "txt": "options", "x": "/html/body/div/div/div/article/div[5]/div/pre/code/span[46]"}, {"position_id": 1711, "txt": "=", "x": "/html/body/div/div/div/article/div[5]/div/pre/code/span[47]"}, {"position_id": 1713, "txt": "{", "x": "/html/body/div/div/div/article/div[5]/div/pre/code/span[48]"}, {"position_id": 1715, "txt": "BinaryName", "x": "/html/body/div/div/div/article/div[5]/div/pre/code/span[49]"}, {"position_id": 1717, "txt": "=", "x": "/html/body/div/div/div/article/div[5]/div/pre/code/span[50]"}, {"position_id": 1719, "txt": "\"", "x": "/html/body/div/div/div/article/div[5]/div/pre/code/span[51]"}, {"position_id": 1721, "txt": "${", "x": "/html/body/div/div/div/article/div[5]/div/pre/code/span[52]"}, {"position_id": 1723, "txt": "pkgs", "x": "/html/body/div/div/div/article/div[5]/div/pre/code/span[53]"}, {"position_id": 1725, "txt": ".", "x": "/html/body/div/div/div/article/div[5]/div/pre/code/span[54]"}, {"position_id": 1727, "txt": "nvidia-container-toolkit", "x": "/html/body/div/div/div/article/div[5]/div/pre/code/span[55]"}, {"position_id": 1729, "txt": "}", "x": "/html/body/div/div/div/article/div[5]/div/pre/code/span[56]"}, {"position_id": 1731, "txt": "/bin/nvidia-container-runtime\"", "x": "/html/body/div/div/div/article/div[5]/div/pre/code/span[57]"}, {"position_id": 1733, "txt": ";", "x": "/html/body/div/div/div/article/div[5]/div/pre/code/span[58]"}, {"position_id": 1735, "txt": "};", "x": "/html/body/div/div/div/article/div[5]/div/pre/code/span[59]"}, {"position_id": 1737, "txt": "};", "x": "/html/body/div/div/div/article/div[5]/div/pre/code/span[60]"}, {"position_id": 1739, "txt": "};", "x": "/html/body/div/div/div/article/div[5]/div/pre/code/span[61]"}, {"position_id": 1741, "txt": "};", "x": "/html/body/div/div/div/article/div[5]/div/pre/code/span[62]"}, {"position_id": 1743, "txt": "};", "x": "/html/body/div/div/div/article/div[5]/div/pre/code/span[63]"}, {"position_id": 1745, "txt": "};", "x": "/html/body/div/div/div/article/div[5]/div/pre/code/span[64]"}, {"position_id": 1747, "txt": "};", "x": "/html/body/div/div/div/article/div[5]/div/pre/code/span[65]"}, {"position_id": 1749, "txt": "};", "x": "/html/body/div/div/div/article/div[5]/div/pre/code/span[66]"}]}, "tags": [], "label": "code", "web_segment_id": 44, "global_sentence_id": 115, "edu_l1_label": "EDU_O"}, {"txt": "nvidia-container-toolkit", "language": "english", "position": {"atoms": [{"position_id": 1751, "txt": "nvidia-container-toolkit", "x": "/html/body/div/div/div/article/h3[2]"}]}, "tags": ["h3"], "label": "title2", "web_segment_id": 45, "global_sentence_id": 116, "edu_l1_label": "IOS"}, {"txt": "We just mentioned CRI above.", "language": "english", "position": {"atoms": [{"position_id": 1753, "txt": "We just mentioned CRI above.", "x": "/html/body/div/div/div/article/p[23]"}]}, "tags": [], "label": "content", "web_segment_id": 46, "global_sentence_id": 117, "edu_l1_label": "IOS"}, {"txt": " One of the jobs of nvidia-container-toolkit is facilitating the CRI interface between Kubernetes’ CRI and the container runtime.", "language": "english", "position": {"atoms": [{"position_id": 1754, "txt": " One of the jobs of ", "x": "/html/body/div/div/div/article/p[23]"}, {"position_id": 1756, "txt": "nvidia-container-toolkit", "x": "/html/body/div/div/div/article/p[23]/a[1]"}, {"position_id": 1758, "txt": " is facilitating the CRI interface between Kubernetes’ CRI and the container runtime.", "x": "/html/body/div/div/div/article/p[23]/a[1]"}]}, "tags": [], "label": "content", "web_segment_id": 46, "global_sentence_id": 118, "edu_l1_label": "IOS"}, {"txt": " What is a container runtime?", "language": "english", "position": {"atoms": [{"position_id": 1759, "txt": " What is a container runtime?", "x": "/html/body/div/div/div/article/p[23]/a[1]"}]}, "tags": [], "label": "content", "web_segment_id": 46, "global_sentence_id": 119, "edu_l1_label": "IOS"}, {"txt": " Usually, it’s a runtime provided as a command line tool to launch an isolated container process by creating new Linux namespaces and mounting host paths into the container.", "language": "english", "position": {"atoms": [{"position_id": 1760, "txt": " Usually, it’s a runtime provided as a command line tool to launch an isolated container process by creating new ", "x": "/html/body/div/div/div/article/p[23]/a[1]"}, {"position_id": 1762, "txt": "Linux namespaces", "x": "/html/body/div/div/div/article/p[23]/a[2]"}, {"position_id": 1764, "txt": " and mounting host paths into the container.", "x": "/html/body/div/div/div/article/p[23]/a[2]"}]}, "tags": [], "label": "content", "web_segment_id": 46, "global_sentence_id": 120, "edu_l1_label": "IOS"}, {"txt": " The most commonly used ones are runc and crun.", "language": "english", "position": {"atoms": [{"position_id": 1765, "txt": " The most commonly used ones are ", "x": "/html/body/div/div/div/article/p[23]/a[2]"}, {"position_id": 1767, "txt": "runc", "x": "/html/body/div/div/div/article/p[23]/a[3]"}, {"position_id": 1769, "txt": " and ", "x": "/html/body/div/div/div/article/p[23]/a[3]"}, {"position_id": 1771, "txt": "crun", "x": "/html/body/div/div/div/article/p[23]/a[4]"}, {"position_id": 1773, "txt": ".", "x": "/html/body/div/div/div/article/p[23]/a[4]"}]}, "tags": [], "label": "content", "web_segment_id": 46, "global_sentence_id": 121, "edu_l1_label": "IOS"}, {"txt": "The nvidia-container-toolkit ship its own container runtime, the nvidia-container-runtime executable.", "language": "english", "position": {"atoms": [{"position_id": 1775, "txt": "The nvidia-container-toolkit ship its own container runtime, the ", "x": "/html/body/div/div/div/article/p[24]"}, {"position_id": 1777, "txt": "nvidia-container-runtime", "x": "/html/body/div/div/div/article/p[24]/code[1]"}, {"position_id": 1779, "txt": " executable.", "x": "/html/body/div/div/div/article/p[24]/code[1]"}]}, "tags": [], "label": "content", "web_segment_id": 47, "global_sentence_id": 122, "edu_l1_label": "IOS"}, {"txt": " It may sound confusing.", "language": "english", "position": {"atoms": [{"position_id": 1780, "txt": " It may sound confusing.", "x": "/html/body/div/div/div/article/p[24]/code[1]"}]}, "tags": [], "label": "content", "web_segment_id": 47, "global_sentence_id": 123, "edu_l1_label": "IOS"}, {"txt": " Why did I say nvidia-container-runtime is a container runtime but also facilitates the CRI interface between Kubernetes and the container runtime?", "language": "english", "position": {"atoms": [{"position_id": 1781, "txt": " Why did I say ", "x": "/html/body/div/div/div/article/p[24]/code[1]"}, {"position_id": 1783, "txt": "nvidia-container-runtime", "x": "/html/body/div/div/div/article/p[24]/code[2]"}, {"position_id": 1785, "txt": " is a container runtime but also facilitates the CRI interface between Kubernetes and the container runtime?", "x": "/html/body/div/div/div/article/p[24]/code[2]"}]}, "tags": [], "label": "content", "web_segment_id": 47, "global_sentence_id": 124, "edu_l1_label": "IOS"}, {"txt": " Well, the thing is, nvidia-container-runtime doesn’t implement the full-blown container runtime.", "language": "english", "position": {"atoms": [{"position_id": 1786, "txt": " Well, the thing is, ", "x": "/html/body/div/div/div/article/p[24]/code[2]"}, {"position_id": 1788, "txt": "nvidia-container-runtime", "x": "/html/body/div/div/div/article/p[24]/code[3]"}, {"position_id": 1790, "txt": " doesn’t implement the full-blown container runtime.", "x": "/html/body/div/div/div/article/p[24]/code[3]"}]}, "tags": [], "label": "content", "web_segment_id": 47, "global_sentence_id": 125, "edu_l1_label": "IOS"}, {"txt": "It’s a thin layer on top of other lower-level runtime implementations such as runc or crun.", "language": "english", "position": {"atoms": [{"position_id": 1792, "txt": "It’s a thin layer on top of other lower-level runtime implementations such as runc or crun.", "x": "/html/body/div/div/div/article/p[25]"}]}, "tags": [], "label": "content", "web_segment_id": 48, "global_sentence_id": 126, "edu_l1_label": "IOS"}, {"txt": " When Containerd or CRI-O invokes Nvidia’s CRI, the OCI containers’ spect will be modified to expose the needed Nvidia drivers, devices, and executables like nvidia-smi into the container.", "language": "english", "position": {"atoms": [{"position_id": 1793, "txt": " When Containerd or CRI-O invokes Nvidia’s CRI, the OCI containers’ spect will be modified to expose the needed Nvidia drivers, devices, and executables like ", "x": "/html/body/div/div/div/article/p[25]"}, {"position_id": 1795, "txt": "nvidia-smi", "x": "/html/body/div/div/div/article/p[25]/code"}, {"position_id": 1797, "txt": " into the container.", "x": "/html/body/div/div/div/article/p[25]/code"}]}, "tags": [], "label": "content", "web_segment_id": 48, "global_sentence_id": 127, "edu_l1_label": "IOS"}, {"txt": " Once the spec file is modified, it will call the underlying crun or runc to do the actual Linux namespace management and all the other container good stuff.", "language": "english", "position": {"atoms": [{"position_id": 1798, "txt": " Once the spec file is modified, it will call the underlying crun or runc to do the actual Linux namespace management and all the other container good stuff.", "x": "/html/body/div/div/div/article/p[25]/code"}]}, "tags": [], "label": "content", "web_segment_id": 48, "global_sentence_id": 128, "edu_l1_label": "IOS"}, {"txt": " So, it’s like a proxy (or some may prefer to call it a shim) between the CRI and the runtime, performing a few modifications to the traffic passing through.", "language": "english", "position": {"atoms": [{"position_id": 1799, "txt": " So, it’s like a proxy (or some may prefer to call it a shim) between the CRI and the runtime, ", "x": "/html/body/div/div/div/article/p[25]/code"}, {"position_id": 1801, "txt": "performing a few modifications", "x": "/html/body/div/div/div/article/p[25]/a"}, {"position_id": 1803, "txt": " to the traffic passing through.", "x": "/html/body/div/div/div/article/p[25]/a"}]}, "tags": [], "label": "content", "web_segment_id": 48, "global_sentence_id": 129, "edu_l1_label": "IOS"}, {"txt": "<img img", "language": "english", "position": {"atoms": [{"position_id": 446, "txt": "<img img", "x": "/html/body/div/div/div/article/figure[7]/img"}]}, "tags": ["img"], "label": "figure", "web_segment_id": 2, "global_sentence_id": 130, "edu_l1_label": "EDU_O"}, {"txt": "Screenshot shows Containerd passes the original OCI spec file config.json to nvidia-container-runtime, and then nvidia-container-runtime modifies the config.json file and passes it along to runc", "language": "english", "position": {"atoms": [{"position_id": 1805, "txt": "Screenshot shows Containerd passes the original OCI spec file config.json to nvidia-container-runtime, and then nvidia-container-runtime modifies the config.json file and passes it along to runc", "x": "/html/body/div/div/div/article/figure[7]/figcaption/p"}]}, "tags": ["figcaption", "img"], "label": "figure_title", "web_segment_id": 49, "global_sentence_id": 131, "edu_l1_label": "EDU_O"}, {"txt": "Other than the container runtime, the nvidia-container-toolkit project also provides utility tools for generating various configurations.", "language": "english", "position": {"atoms": [{"position_id": 1807, "txt": "Other than the container runtime, the nvidia-container-toolkit project also provides utility tools for generating various configurations.", "x": "/html/body/div/div/div/article/p[26]"}]}, "tags": [], "label": "content", "web_segment_id": 50, "global_sentence_id": 132, "edu_l1_label": "IOS"}, {"txt": " Such as this one for generating container runtime configuration for Containerd:", "language": "english", "position": {"atoms": [{"position_id": 1808, "txt": " Such as this one for generating container runtime configuration for Containerd:", "x": "/html/body/div/div/div/article/p[26]"}]}, "tags": [], "label": "content", "web_segment_id": 50, "global_sentence_id": 133, "edu_l1_label": "IOS"}, {"txt": "nvidia-ctk runtime configure --runtime=containerd", "language": "english", "position": {"atoms": [{"position_id": 1810, "txt": "nvidia-ctk runtime configure ", "x": "/html/body/div/div/div/article/div[6]/div/pre/code"}, {"position_id": 1812, "txt": "--runtime", "x": "/html/body/div/div/div/article/div[6]/div/pre/code/span[1]"}, {"position_id": 1814, "txt": "=", "x": "/html/body/div/div/div/article/div[6]/div/pre/code/span[2]"}, {"position_id": 1816, "txt": "containerd\n", "x": "/html/body/div/div/div/article/div[6]/div/pre/code/span[2]"}]}, "tags": [], "label": "code", "web_segment_id": 51, "global_sentence_id": 134, "edu_l1_label": "IOS"}, {"txt": "libnvidia-container", "language": "english", "position": {"atoms": [{"position_id": 1818, "txt": "libnvidia-container", "x": "/html/body/div/div/div/article/h3[3]"}]}, "tags": ["h3"], "label": "title2", "web_segment_id": 52, "global_sentence_id": 135, "edu_l1_label": "IOS"}, {"txt": "There are some confusing parts about this repository.", "language": "english", "position": {"atoms": [{"position_id": 1820, "txt": "There are some confusing parts about this repository.", "x": "/html/body/div/div/div/article/p[27]"}]}, "tags": [], "label": "content", "web_segment_id": 53, "global_sentence_id": 136, "edu_l1_label": "IOS"}, {"txt": " While its name says libnvidia-container, they don’t use it as a library like all other projects with lib prefixes would usually do; at last, from the perspective of the Kubernetes plugin.", "language": "english", "position": {"atoms": [{"position_id": 1821, "txt": " While its name says ", "x": "/html/body/div/div/div/article/p[27]"}, {"position_id": 1823, "txt": "libnvidia-container", "x": "/html/body/div/div/div/article/p[27]/a[1]"}, {"position_id": 1825, "txt": ", they don’t use it as a library like all other projects with lib prefixes would usually do; at last, from the perspective of the Kubernetes plugin.", "x": "/html/body/div/div/div/article/p[27]/a[1]"}]}, "tags": [], "label": "content", "web_segment_id": 53, "global_sentence_id": 137, "edu_l1_label": "IOS"}, {"txt": " Instead, it provides an executable, nvidia-container-cli.", "language": "english", "position": {"atoms": [{"position_id": 1826, "txt": " Instead, it provides an executable, ", "x": "/html/body/div/div/div/article/p[27]/a[1]"}, {"position_id": 1828, "txt": "nvidia-container-cli", "x": "/html/body/div/div/div/article/p[27]/code[1]"}, {"position_id": 1830, "txt": ".", "x": "/html/body/div/div/div/article/p[27]/code[1]"}]}, "tags": [], "label": "content", "web_segment_id": 53, "global_sentence_id": 138, "edu_l1_label": "IOS"}, {"txt": " The main purpose of this executable is to collect and provide information about Nvidia drivers, devices, and executables to be injected into the container environment.", "language": "english", "position": {"atoms": [{"position_id": 1831, "txt": " The main purpose of this executable is to collect and provide information about Nvidia drivers, devices, and executables to be injected into the container environment.", "x": "/html/body/div/div/div/article/p[27]/code[1]"}]}, "tags": [], "label": "content", "web_segment_id": 53, "global_sentence_id": 139, "edu_l1_label": "IOS"}, {"txt": " Usually invoked directly from the nvidia-container-runtime’s hook command.", "language": "english", "position": {"atoms": [{"position_id": 1832, "txt": " Usually ", "x": "/html/body/div/div/div/article/p[27]/code[1]"}, {"position_id": 1834, "txt": "invoked directly", "x": "/html/body/div/div/div/article/p[27]/a[2]"}, {"position_id": 1836, "txt": " from the ", "x": "/html/body/div/div/div/article/p[27]/a[2]"}, {"position_id": 1838, "txt": "nvidia-container-runtime", "x": "/html/body/div/div/div/article/p[27]/code[2]"}, {"position_id": 1840, "txt": "’s hook command.", "x": "/html/body/div/div/div/article/p[27]/code[2]"}]}, "tags": [], "label": "content", "web_segment_id": 53, "global_sentence_id": 140, "edu_l1_label": "IOS"}, {"txt": " I guess they need to use the same logic somewhere in Nvidia’s container-relative projects, and that might be why they need to make it a library.", "language": "english", "position": {"atoms": [{"position_id": 1841, "txt": " I guess they need to use the same logic somewhere in Nvidia’s container-relative projects, and that might be why they need to make it a library.", "x": "/html/body/div/div/div/article/p[27]/code[2]"}]}, "tags": [], "label": "content", "web_segment_id": 53, "global_sentence_id": 141, "edu_l1_label": "IOS"}, {"txt": "This command-line tool is used when using Nvidia as container runtime directly without CDI support.", "language": "english", "position": {"atoms": [{"position_id": 1843, "txt": "This command-line tool is used when using Nvidia as container runtime directly without CDI support.", "x": "/html/body/div/div/div/article/p[28]"}]}, "tags": [], "label": "content", "web_segment_id": 54, "global_sentence_id": 142, "edu_l1_label": "IOS"}, {"txt": " Otherwise, it appears that if we take the CDI route, this command-line tool is not used at all.", "language": "english", "position": {"atoms": [{"position_id": 1844, "txt": " Otherwise, it appears that if we take the CDI route, this command-line tool is not used at all.", "x": "/html/body/div/div/div/article/p[28]"}]}, "tags": [], "label": "content", "web_segment_id": 54, "global_sentence_id": 143, "edu_l1_label": "IOS"}, {"txt": "Container Device Interface", "language": "english", "position": {"atoms": [{"position_id": 1846, "txt": "Container Device Interface", "x": "/html/body/div/div/div/article/h3[4]"}]}, "tags": ["h3"], "label": "title2", "web_segment_id": 55, "global_sentence_id": 144, "edu_l1_label": "IOS"}, {"txt": "Similar to CRI, it’s also a well-defined plugin for Kubernetes.", "language": "english", "position": {"atoms": [{"position_id": 1848, "txt": "Similar to CRI, it’s also a well-defined plugin for Kubernetes.", "x": "/html/body/div/div/div/article/p[29]"}]}, "tags": [], "label": "content", "web_segment_id": 56, "global_sentence_id": 145, "edu_l1_label": "IOS"}, {"txt": " Instead of providing container runtime functionalities, it offers particular device resources to expose to the container environment.", "language": "english", "position": {"atoms": [{"position_id": 1849, "txt": " Instead of providing container runtime functionalities, it offers particular device resources to expose to the container environment.", "x": "/html/body/div/div/div/article/p[29]"}]}, "tags": [], "label": "content", "web_segment_id": 56, "global_sentence_id": 146, "edu_l1_label": "IOS"}, {"txt": " With Nvidia’s CDI (Container Device Interface) implementation, we can then expose GPU devices as resources in the container.", "language": "english", "position": {"atoms": [{"position_id": 1850, "txt": " With Nvidia’s ", "x": "/html/body/div/div/div/article/p[29]"}, {"position_id": 1852, "txt": "CDI (Container Device Interface)", "x": "/html/body/div/div/div/article/p[29]/a"}, {"position_id": 1854, "txt": " implementation, we can then expose GPU devices as resources in the container.", "x": "/html/body/div/div/div/article/p[29]/a"}]}, "tags": [], "label": "content", "web_segment_id": 56, "global_sentence_id": 147, "edu_l1_label": "IOS"}, {"txt": "In addition to Kubernetes, a container CLI tool like Podman can also use proper CDI configuration to have an Nvidia GPU inside the container.", "language": "english", "position": {"atoms": [{"position_id": 1856, "txt": "In addition to Kubernetes, a container CLI tool like Podman can also use proper CDI configuration to have an Nvidia GPU inside the container.", "x": "/html/body/div/div/div/article/p[30]"}]}, "tags": [], "label": "content", "web_segment_id": 57, "global_sentence_id": 148, "edu_l1_label": "IOS"}, {"txt": "$ podman run --device nvidia.com/gpu=all alpine nvidia-smiSat Mar  1 23:", "language": "english", "position": {"atoms": [{"position_id": 1858, "txt": "$ podman run --device nvidia.com/gpu=all alpine nvidia-smi\nSat Mar  1 23:", "x": "/html/body/div/div/div/article/div[7]/div/pre/code"}]}, "tags": [], "label": "code", "web_segment_id": 58, "global_sentence_id": 149, "edu_l1_label": "EDU_O"}, {"txt": "51:", "language": "english", "position": {"atoms": [{"position_id": 1859, "txt": "51:", "x": "/html/body/div/div/div/article/div[7]/div/pre/code"}]}, "tags": [], "label": "code", "web_segment_id": 58, "global_sentence_id": 150, "edu_l1_label": "EDU_O"}, {"txt": "00 2025+-----------------------------------------------------------------------------------------+| NVIDIA-SMI 565.77                 Driver Version:", "language": "english", "position": {"atoms": [{"position_id": 1860, "txt": "00 2025\n+-----------------------------------------------------------------------------------------+\n| NVIDIA-SMI 565.77                 Driver Version:", "x": "/html/body/div/div/div/article/div[7]/div/pre/code"}]}, "tags": [], "label": "code", "web_segment_id": 58, "global_sentence_id": 151, "edu_l1_label": "EDU_O"}, {"txt": " 565.77         CUDA Version:", "language": "english", "position": {"atoms": [{"position_id": 1861, "txt": " 565.77         CUDA Version:", "x": "/html/body/div/div/div/article/div[7]/div/pre/code"}]}, "tags": [], "label": "code", "web_segment_id": 58, "global_sentence_id": 152, "edu_l1_label": "EDU_O"}, {"txt": " 12.7     ||-----------------------------------------+------------------------+----------------------+| GPU  Name                 Persistence-M | Bus-Id          Disp.A | Volatile Uncorr.", "language": "english", "position": {"atoms": [{"position_id": 1862, "txt": " 12.7     |\n|-----------------------------------------+------------------------+----------------------+\n| GPU  Name                 Persistence-M | Bus-Id          Disp.A | Volatile Uncorr.", "x": "/html/body/div/div/div/article/div[7]/div/pre/code"}]}, "tags": [], "label": "code", "web_segment_id": 58, "global_sentence_id": 153, "edu_l1_label": "EDU_O"}, {"txt": " ECC || Fan  Temp   Perf          Pwr:", "language": "english", "position": {"atoms": [{"position_id": 1863, "txt": " ECC |\n| Fan  Temp   Perf          Pwr:", "x": "/html/body/div/div/div/article/div[7]/div/pre/code"}]}, "tags": [], "label": "code", "web_segment_id": 58, "global_sentence_id": 154, "edu_l1_label": "EDU_O"}, {"txt": "Usage/Cap |           Memory-Usage | GPU-Util  Compute M. ||                                         |                        |               MIG M. ||=========================================+========================+======================||   0  NVIDIA GeForce RTX 2080 Ti     Off |   00000000:", "language": "english", "position": {"atoms": [{"position_id": 1864, "txt": "Usage/Cap |           Memory-Usage | GPU-Util  Compute M. |\n|                                         |                        |               MIG M. |\n|=========================================+========================+======================|\n|   0  NVIDIA GeForce RTX 2080 Ti     Off |   00000000:", "x": "/html/body/div/div/div/article/div[7]/div/pre/code"}]}, "tags": [], "label": "code", "web_segment_id": 58, "global_sentence_id": 155, "edu_l1_label": "EDU_O"}, {"txt": "0A:", "language": "english", "position": {"atoms": [{"position_id": 1865, "txt": "0A:", "x": "/html/body/div/div/div/article/div[7]/div/pre/code"}]}, "tags": [], "label": "code", "web_segment_id": 58, "global_sentence_id": 156, "edu_l1_label": "EDU_O"}, {"txt": "00.0  On |                  N/A || 46%   57C    P2            103W /  260W |    3998MiB /  11264MiB |     33%      Default ||                                         |                        |                  N/A |+-----------------------------------------+------------------------+----------------------++-----------------------------------------------------------------------------------------+| Processes:", "language": "english", "position": {"atoms": [{"position_id": 1866, "txt": "00.0  On |                  N/A |\n| 46%   57C    P2            103W /  260W |    3998MiB /  11264MiB |     33%      Default |\n|                                         |                        |                  N/A |\n+-----------------------------------------+------------------------+----------------------+\n\n+-----------------------------------------------------------------------------------------+\n| Processes:", "x": "/html/body/div/div/div/article/div[7]/div/pre/code"}]}, "tags": [], "label": "code", "web_segment_id": 58, "global_sentence_id": 157, "edu_l1_label": "EDU_O"}, {"txt": "                                                                              ||  GPU   GI   CI        PID   Type   Process name                              GPU Memory ||        ID   ID                                                               Usage      ||=========================================================================================||  No running processes found                                                             |+-----------------------------------------------------------------------------------------+", "language": "english", "position": {"atoms": [{"position_id": 1867, "txt": "                                                                              |\n|  GPU   GI   CI        PID   Type   Process name                              GPU Memory |\n|        ID   ID                                                               Usage      |\n|=========================================================================================|\n|  No running processes found                                                             |\n+-----------------------------------------------------------------------------------------+\n", "x": "/html/body/div/div/div/article/div[7]/div/pre/code"}]}, "tags": [], "label": "code", "web_segment_id": 58, "global_sentence_id": 158, "edu_l1_label": "EDU_O"}, {"txt": "The --device argument with nvidia.com/gpu=all value tells podman to look for the resource (device) type nvidia.com/gpu, and we want all of it exposed inside the container.", "language": "english", "position": {"atoms": [{"position_id": 1869, "txt": "The ", "x": "/html/body/div/div/div/article/p[31]"}, {"position_id": 1871, "txt": "--device", "x": "/html/body/div/div/div/article/p[31]/code[1]"}, {"position_id": 1873, "txt": " argument with ", "x": "/html/body/div/div/div/article/p[31]/code[1]"}, {"position_id": 1875, "txt": "nvidia.com/gpu=all", "x": "/html/body/div/div/div/article/p[31]/code[2]"}, {"position_id": 1877, "txt": " value tells podman to look for the resource (device) type ", "x": "/html/body/div/div/div/article/p[31]/code[2]"}, {"position_id": 1879, "txt": "nvidia.com/gpu", "x": "/html/body/div/div/div/article/p[31]/code[3]"}, {"position_id": 1881, "txt": ", and we want all of it exposed inside the container.", "x": "/html/body/div/div/div/article/p[31]/code[3]"}]}, "tags": [], "label": "content", "web_segment_id": 59, "global_sentence_id": 159, "edu_l1_label": "IOS"}, {"txt": "k8s-device-plugin", "language": "english", "position": {"atoms": [{"position_id": 1883, "txt": "k8s-device-plugin", "x": "/html/body/div/div/div/article/h3[5]"}]}, "tags": ["h3"], "label": "title2", "web_segment_id": 60, "global_sentence_id": 160, "edu_l1_label": "IOS"}, {"txt": "With nvidia-container-toolkit and libnvidia-container, now it’s possible to run a container with Nvidia GPU exposed inside a container, but then we still need a way to let Kubernetes know there’s nvidia.com/gpu such resource available for pods to use, and we need to know which nodes have them, how many are available, how many are in use, and what types.", "language": "english", "position": {"atoms": [{"position_id": 1885, "txt": "With nvidia-container-toolkit and libnvidia-container, now it’s possible to run a container with Nvidia GPU exposed inside a container, but then we still need a way to let Kubernetes know there’s ", "x": "/html/body/div/div/div/article/p[32]"}, {"position_id": 1887, "txt": "nvidia.com/gpu", "x": "/html/body/div/div/div/article/p[32]/code"}, {"position_id": 1889, "txt": " such resource available for pods to use, and we need to know which nodes have them, how many are available, how many are in use, and what types.", "x": "/html/body/div/div/div/article/p[32]/code"}]}, "tags": [], "label": "content", "web_segment_id": 61, "global_sentence_id": 161, "edu_l1_label": "IOS"}, {"txt": " The k8s-device-plugin is here to solve those problems.", "language": "english", "position": {"atoms": [{"position_id": 1890, "txt": " The ", "x": "/html/body/div/div/div/article/p[32]/code"}, {"position_id": 1892, "txt": "k8s-device-plugin", "x": "/html/body/div/div/div/article/p[32]/a"}, {"position_id": 1894, "txt": " is here to solve those problems.", "x": "/html/body/div/div/div/article/p[32]/a"}]}, "tags": [], "label": "content", "web_segment_id": 61, "global_sentence_id": 162, "edu_l1_label": "IOS"}, {"txt": "The k8s-device-plugin deployed in the Kubernetes cluster has DaemonSet to check for available Nvidia GPU devices on each node, report on them, and manage that information with the Kubernetes API server.", "language": "english", "position": {"atoms": [{"position_id": 1896, "txt": "The k8s-device-plugin deployed in the Kubernetes cluster has ", "x": "/html/body/div/div/div/article/p[33]"}, {"position_id": 1898, "txt": "DaemonSet", "x": "/html/body/div/div/div/article/p[33]/a"}, {"position_id": 1900, "txt": " to check for available Nvidia GPU devices on each node, report on them, and manage that information with the Kubernetes API server.", "x": "/html/body/div/div/div/article/p[33]/a"}]}, "tags": [], "label": "content", "web_segment_id": 62, "global_sentence_id": 163, "edu_l1_label": "IOS"}, {"txt": "Dig this hole first - build nix-playground for hacking and patching any source code easily", "language": "english", "position": {"atoms": [{"position_id": 1902, "txt": "Dig this hole first - build nix-playground for hacking and patching any source code easily", "x": "/html/body/div/div/div/article/h2[4]"}]}, "tags": ["h2"], "label": "title1", "web_segment_id": 63, "global_sentence_id": 164, "edu_l1_label": "BOS"}, {"txt": "Now we have the background knowledge about what’s what in the context of Nvidia GPU on Kubernetes.", "language": "english", "position": {"atoms": [{"position_id": 1904, "txt": "Now we have the background knowledge about what’s what in the context of Nvidia GPU on Kubernetes.", "x": "/html/body/div/div/div/article/p[34]"}]}, "tags": [], "label": "content", "web_segment_id": 64, "global_sentence_id": 165, "edu_l1_label": "IOS"}, {"txt": " Do you think it’s about time to discuss adding Nvidia GPU support to your bare-metal Kubernetes cluster?", "language": "english", "position": {"atoms": [{"position_id": 1905, "txt": " Do you think it’s about time to discuss adding Nvidia GPU support to your bare-metal Kubernetes cluster?", "x": "/html/body/div/div/div/article/p[34]"}]}, "tags": [], "label": "content", "web_segment_id": 64, "global_sentence_id": 166, "edu_l1_label": "IOS"}, {"txt": " Well, not so fast.", "language": "english", "position": {"atoms": [{"position_id": 1906, "txt": " Well, not so fast.", "x": "/html/body/div/div/div/article/p[34]"}]}, "tags": [], "label": "content", "web_segment_id": 64, "global_sentence_id": 167, "edu_l1_label": "IOS"}, {"txt": " As I thought it shouldn’t be too hard, just deploy the k8s-device-plugin as a helm chart, generate and modify configurations as described in Nvidia’s official documents, and it should be good to go, right?", "language": "english", "position": {"atoms": [{"position_id": 1907, "txt": " As I thought it shouldn’t be too hard, just deploy the k8s-device-plugin as ", "x": "/html/body/div/div/div/article/p[34]"}, {"position_id": 1909, "txt": "a helm chart", "x": "/html/body/div/div/div/article/p[34]/a"}, {"position_id": 1911, "txt": ", generate and modify configurations as described in Nvidia’s official documents, and it should be good to go, right?", "x": "/html/body/div/div/div/article/p[34]/a"}]}, "tags": [], "label": "content", "web_segment_id": 64, "global_sentence_id": 168, "edu_l1_label": "IOS"}, {"txt": " Unfortunately, I saw many errors immediately right after I did so.", "language": "english", "position": {"atoms": [{"position_id": 1912, "txt": " Unfortunately, I saw many errors immediately right after I did so.", "x": "/html/body/div/div/div/article/p[34]/a"}]}, "tags": [], "label": "content", "web_segment_id": 64, "global_sentence_id": 169, "edu_l1_label": "IOS"}, {"txt": " Like this one from nvdp-nvidia-device-plugin DaemonSet:", "language": "english", "position": {"atoms": [{"position_id": 1913, "txt": " Like this one from ", "x": "/html/body/div/div/div/article/p[34]/a"}, {"position_id": 1915, "txt": "nvdp-nvidia-device-plugin", "x": "/html/body/div/div/div/article/p[34]/code"}, {"position_id": 1917, "txt": " DaemonSet:", "x": "/html/body/div/div/div/article/p[34]/code"}]}, "tags": [], "label": "content", "web_segment_id": 64, "global_sentence_id": 170, "edu_l1_label": "IOS"}, {"txt": "symbol lookup error:", "language": "english", "position": {"atoms": [{"position_id": 1919, "txt": "symbol lookup error:", "x": "/html/body/div/div/div/article/div[8]/div/pre/code"}]}, "tags": [], "label": "content", "web_segment_id": 65, "global_sentence_id": 171, "edu_l1_label": "IOS"}, {"txt": " /nix/store/nqb2ns2d1lahnd5ncwmn6k84qfd7vx2k-glibc-2.40-36/lib/libc.so.6:", "language": "english", "position": {"atoms": [{"position_id": 1920, "txt": " /nix/store/nqb2ns2d1lahnd5ncwmn6k84qfd7vx2k-glibc-2.40-36/lib/libc.so.6:", "x": "/html/body/div/div/div/article/div[8]/div/pre/code"}]}, "tags": [], "label": "content", "web_segment_id": 65, "global_sentence_id": 172, "edu_l1_label": "IOS"}, {"txt": " undefined symbol:", "language": "english", "position": {"atoms": [{"position_id": 1921, "txt": " undefined symbol:", "x": "/html/body/div/div/div/article/div[8]/div/pre/code"}]}, "tags": [], "label": "content", "web_segment_id": 65, "global_sentence_id": 173, "edu_l1_label": "IOS"}, {"txt": " __tunable_is_initialized, version GLIBC_PRIVATE", "language": "english", "position": {"atoms": [{"position_id": 1922, "txt": " __tunable_is_initialized, version GLIBC_PRIVATE\n", "x": "/html/body/div/div/div/article/div[8]/div/pre/code"}]}, "tags": [], "label": "content", "web_segment_id": 65, "global_sentence_id": 174, "edu_l1_label": "IOS"}, {"txt": "It turns out we need to dig deeper 😅.", "language": "english", "position": {"atoms": [{"position_id": 1924, "txt": "It turns out we need to dig deeper 😅.", "x": "/html/body/div/div/div/article/p[35]"}]}, "tags": [], "label": "content", "web_segment_id": 66, "global_sentence_id": 175, "edu_l1_label": "IOS"}, {"txt": "<img img", "language": "english", "position": {"atoms": [{"position_id": 491, "txt": "<img img", "x": "/html/body/div/div/div/article/figure[8]/img"}]}, "tags": ["img"], "label": "figure", "web_segment_id": 2, "global_sentence_id": 176, "edu_l1_label": "EDU_O"}, {"txt": "The We Need To Go Deeper meme from the Inception movie", "language": "english", "position": {"atoms": [{"position_id": 1926, "txt": "The ", "x": "/html/body/div/div/div/article/figure[8]/figcaption/p"}, {"position_id": 1928, "txt": "We Need To Go Deeper", "x": "/html/body/div/div/div/article/figure[8]/figcaption/p/a"}, {"position_id": 1930, "txt": " meme from the Inception movie", "x": "/html/body/div/div/div/article/figure[8]/figcaption/p/a"}]}, "tags": ["figcaption", "img"], "label": "figure_title", "web_segment_id": 67, "global_sentence_id": 177, "edu_l1_label": "EDU_O"}, {"txt": "I am the type of engineer who enjoys digging rabbit holes most of the time and learns a lot from doing so.", "language": "english", "position": {"atoms": [{"position_id": 1932, "txt": "I am the type of engineer who enjoys digging rabbit holes most of the time and learns a lot from doing so.", "x": "/html/body/div/div/div/article/p[36]"}]}, "tags": [], "label": "content", "web_segment_id": 68, "global_sentence_id": 178, "edu_l1_label": "IOS"}, {"txt": " Many people don’t understand software engineering, so I have said countless times that reading code is way harder than writing code.", "language": "english", "position": {"atoms": [{"position_id": 1933, "txt": " Many people don’t understand software engineering, so I have said countless times that reading code is way harder than writing code.", "x": "/html/body/div/div/div/article/p[36]"}]}, "tags": [], "label": "content", "web_segment_id": 68, "global_sentence_id": 179, "edu_l1_label": "IOS"}, {"txt": " In the era of AI, generating code makes it even harder to read.", "language": "english", "position": {"atoms": [{"position_id": 1934, "txt": " In the era of AI, generating code makes it even harder to read.", "x": "/html/body/div/div/div/article/p[36]"}]}, "tags": [], "label": "content", "web_segment_id": 68, "global_sentence_id": 180, "edu_l1_label": "IOS"}, {"txt": " And reading source code upstream is critical for solving hard problems.", "language": "english", "position": {"atoms": [{"position_id": 1935, "txt": " And reading source code upstream is critical for solving hard problems.", "x": "/html/body/div/div/div/article/p[36]"}]}, "tags": [], "label": "content", "web_segment_id": 68, "global_sentence_id": 181, "edu_l1_label": "IOS"}, {"txt": " Sometimes, you are lucky, and there are easy ways to work around problems, but sometimes, there’s no easy way around but to find the root cause.", "language": "english", "position": {"atoms": [{"position_id": 1936, "txt": " Sometimes, you are lucky, and there are easy ways to work around problems, but sometimes, there’s no easy way around but to find the root cause.", "x": "/html/body/div/div/div/article/p[36]"}]}, "tags": [], "label": "content", "web_segment_id": 68, "global_sentence_id": 182, "edu_l1_label": "IOS"}, {"txt": "An error like this one is very tricky because the executable crashes immediately after it starts the container process in the Linux namespace.", "language": "english", "position": {"atoms": [{"position_id": 1938, "txt": "An error like this one is very tricky because the executable crashes immediately after it starts the container process in the Linux namespace.", "x": "/html/body/div/div/div/article/p[37]"}]}, "tags": [], "label": "content", "web_segment_id": 69, "global_sentence_id": 183, "edu_l1_label": "IOS"}, {"txt": " Usually, when seeing an error like this, I just exec or debug into the container with a shell to see what’s happening.", "language": "english", "position": {"atoms": [{"position_id": 1939, "txt": " Usually, when seeing an error like this, I just ", "x": "/html/body/div/div/div/article/p[37]"}, {"position_id": 1941, "txt": "exec", "x": "/html/body/div/div/div/article/p[37]/a[1]"}, {"position_id": 1943, "txt": " or ", "x": "/html/body/div/div/div/article/p[37]/a[1]"}, {"position_id": 1945, "txt": "debug", "x": "/html/body/div/div/div/article/p[37]/a[2]"}, {"position_id": 1947, "txt": " into the container with a shell to see what’s happening.", "x": "/html/body/div/div/div/article/p[37]/a[2]"}]}, "tags": [], "label": "content", "web_segment_id": 69, "global_sentence_id": 184, "edu_l1_label": "IOS"}, {"txt": " But it won’t work in this case because all executables crash like that in the container, even for /bin/sh.", "language": "english", "position": {"atoms": [{"position_id": 1948, "txt": " But it won’t work in this case because all executables crash like that in the container, even for ", "x": "/html/body/div/div/div/article/p[37]/a[2]"}, {"position_id": 1950, "txt": "/bin/sh.", "x": "/html/body/div/div/div/article/p[37]/code"}]}, "tags": [], "label": "content", "web_segment_id": 69, "global_sentence_id": 185, "edu_l1_label": "IOS"}, {"txt": "I had no choice but to read the source code of Nvidia’s container projects, and I needed to modify the code to figure out what was going on.", "language": "english", "position": {"atoms": [{"position_id": 1952, "txt": "I had no choice but to read the source code of Nvidia’s container projects, and I needed to modify the code to figure out what was going on.", "x": "/html/body/div/div/div/article/p[38]"}]}, "tags": [], "label": "content", "web_segment_id": 70, "global_sentence_id": 186, "edu_l1_label": "IOS"}, {"txt": " Unfortunately, modifying source code and making a custom build is annoying.", "language": "english", "position": {"atoms": [{"position_id": 1953, "txt": " Unfortunately, modifying source code and making a custom build is annoying.", "x": "/html/body/div/div/div/article/p[38]"}]}, "tags": [], "label": "content", "web_segment_id": 70, "global_sentence_id": 187, "edu_l1_label": "IOS"}, {"txt": " Just to get the project to build could take hours.", "language": "english", "position": {"atoms": [{"position_id": 1954, "txt": " Just to get the project to build could take hours.", "x": "/html/body/div/div/div/article/p[38]"}]}, "tags": [], "label": "content", "web_segment_id": 70, "global_sentence_id": 188, "edu_l1_label": "IOS"}, {"txt": " You need to read the instructions to install the required dependencies.", "language": "english", "position": {"atoms": [{"position_id": 1955, "txt": " You need to read the instructions to install the required dependencies.", "x": "/html/body/div/div/div/article/p[38]"}]}, "tags": [], "label": "content", "web_segment_id": 70, "global_sentence_id": 189, "edu_l1_label": "IOS"}, {"txt": " Sometimes, there are very specific versions of those dependencies or build environments you need to get.", "language": "english", "position": {"atoms": [{"position_id": 1956, "txt": " Sometimes, there are very specific versions of those dependencies or build environments you need to get.", "x": "/html/body/div/div/div/article/p[38]"}]}, "tags": [], "label": "content", "web_segment_id": 70, "global_sentence_id": 190, "edu_l1_label": "IOS"}, {"txt": " Very often, it could take hours or even a few days just to get a project to build, depending on how complex the project is.", "language": "english", "position": {"atoms": [{"position_id": 1957, "txt": " Very often, it could take hours or even a few days just to get a project to build, depending on how complex the project is.", "x": "/html/body/div/div/div/article/p[38]"}]}, "tags": [], "label": "content", "web_segment_id": 70, "global_sentence_id": 191, "edu_l1_label": "IOS"}, {"txt": "I wish there was a tool to help me hack the source code of any open-source projects and build them quickly with my code changes so I could try things out quickly.", "language": "english", "position": {"atoms": [{"position_id": 1959, "txt": "I wish there was a tool to help me hack the source code of any open-source projects and build them quickly with my code changes so I could try things out quickly.", "x": "/html/body/div/div/div/article/p[39]"}]}, "tags": [], "label": "content", "web_segment_id": 71, "global_sentence_id": 192, "edu_l1_label": "IOS"}, {"txt": " Then I realized, okay, why not just build one myself?", "language": "english", "position": {"atoms": [{"position_id": 1960, "txt": " Then I realized, okay, why not just build one myself?", "x": "/html/body/div/div/div/article/p[39]"}]}, "tags": [], "label": "content", "web_segment_id": 71, "global_sentence_id": 193, "edu_l1_label": "IOS"}, {"txt": " Obviously, this is yet another rabbit hole to dig from the original one.", "language": "english", "position": {"atoms": [{"position_id": 1961, "txt": " Obviously, this is yet another rabbit hole to dig from the original one.", "x": "/html/body/div/div/div/article/p[39]"}]}, "tags": [], "label": "content", "web_segment_id": 71, "global_sentence_id": 194, "edu_l1_label": "IOS"}, {"txt": " But if it works, this could be super useful down the road.", "language": "english", "position": {"atoms": [{"position_id": 1962, "txt": " But if it works, this could be super useful down the road.", "x": "/html/body/div/div/div/article/p[39]"}]}, "tags": [], "label": "content", "web_segment_id": 71, "global_sentence_id": 195, "edu_l1_label": "IOS"}, {"txt": " Considering the potential productivity gains and the leverage this tool could have.", "language": "english", "position": {"atoms": [{"position_id": 1963, "txt": " Considering the potential productivity gains and the leverage this tool could have.", "x": "/html/body/div/div/div/article/p[39]"}]}, "tags": [], "label": "content", "web_segment_id": 71, "global_sentence_id": 196, "edu_l1_label": "IOS"}, {"txt": " I’ve decided to context-switch to digging this rabbit hole shortly.", "language": "english", "position": {"atoms": [{"position_id": 1964, "txt": " I’ve decided to context-switch to digging this rabbit hole shortly.", "x": "/html/body/div/div/div/article/p[39]"}]}, "tags": [], "label": "content", "web_segment_id": 71, "global_sentence_id": 197, "edu_l1_label": "IOS"}, {"txt": "I love NixOS and Nixpkgs because they’re a build tree containing almost all major open-source software, from the Linux kernel to a utils command-line tool.", "language": "english", "position": {"atoms": [{"position_id": 1966, "txt": "I love NixOS and Nixpkgs because they’re a build tree containing almost all major open-source software, from the Linux kernel to a utils command-line tool.", "x": "/html/body/div/div/div/article/p[40]"}]}, "tags": [], "label": "content", "web_segment_id": 72, "global_sentence_id": 198, "edu_l1_label": "IOS"}, {"txt": " As far as I know, nothing else like this allows you to build the whole system from end to end under the same framework.", "language": "english", "position": {"atoms": [{"position_id": 1967, "txt": " As far as I know, nothing else like this allows you to build the whole system from end to end under the same framework.", "x": "/html/body/div/div/div/article/p[40]"}]}, "tags": [], "label": "content", "web_segment_id": 72, "global_sentence_id": 199, "edu_l1_label": "IOS"}, {"txt": " Thanks to the nature of this tool, it has the building scripts ready in Nix for all the projects I want to look into, including the nvidia-container-toolkit and libnvidia-container.", "language": "english", "position": {"atoms": [{"position_id": 1968, "txt": " Thanks to the nature of this tool, it has the building scripts ready in Nix for all the projects I want to look into, including the ", "x": "/html/body/div/div/div/article/p[40]"}, {"position_id": 1970, "txt": "nvidia-container-toolkit", "x": "/html/body/div/div/div/article/p[40]/a[1]"}, {"position_id": 1972, "txt": " and ", "x": "/html/body/div/div/div/article/p[40]/a[1]"}, {"position_id": 1974, "txt": "libnvidia-container", "x": "/html/body/div/div/div/article/p[40]/a[2]"}, {"position_id": 1976, "txt": ".", "x": "/html/body/div/div/div/article/p[40]/a[2]"}]}, "tags": [], "label": "content", "web_segment_id": 72, "global_sentence_id": 200, "edu_l1_label": "IOS"}, {"txt": "Since each Nixpkgs package is just the artifact of yet another Nix’s derivation, which describes how to build the project and all its dependencies in a pure data format (ATerm), I should be able to extract the source code from it.", "language": "english", "position": {"atoms": [{"position_id": 1978, "txt": "Since each Nixpkgs package is just the artifact of yet another ", "x": "/html/body/div/div/div/article/p[41]"}, {"position_id": 1980, "txt": "Nix’s derivation", "x": "/html/body/div/div/div/article/p[41]/a[1]"}, {"position_id": 1982, "txt": ", which describes how to build the project and all its dependencies in a ", "x": "/html/body/div/div/div/article/p[41]/a[1]"}, {"position_id": 1984, "txt": "pure data format (ATerm)", "x": "/html/body/div/div/div/article/p[41]/a[2]"}, {"position_id": 1986, "txt": ", I should be able to extract the source code from it.", "x": "/html/body/div/div/div/article/p[41]/a[2]"}]}, "tags": [], "label": "content", "web_segment_id": 73, "global_sentence_id": 201, "edu_l1_label": "IOS"}, {"txt": " With that in mind, I quickly built a prototype to try out, which I called nix-playground.", "language": "english", "position": {"atoms": [{"position_id": 1987, "txt": " With that in mind, I quickly built a prototype to try out, which I called ", "x": "/html/body/div/div/div/article/p[41]/a[2]"}, {"position_id": 1989, "txt": "nix-playground", "x": "/html/body/div/div/div/article/p[41]/a[3]"}, {"position_id": 1991, "txt": ".", "x": "/html/body/div/div/div/article/p[41]/a[3]"}]}, "tags": [], "label": "content", "web_segment_id": 73, "global_sentence_id": 202, "edu_l1_label": "IOS"}, {"txt": " And yes, I open-sourced it under the MIT license.", "language": "english", "position": {"atoms": [{"position_id": 1992, "txt": " And yes, I open-sourced it under the MIT license.", "x": "/html/body/div/div/div/article/p[41]/a[3]"}]}, "tags": [], "label": "content", "web_segment_id": 73, "global_sentence_id": 203, "edu_l1_label": "IOS"}, {"txt": " You can use it as a playground to easily play around and patch any open-source project.", "language": "english", "position": {"atoms": [{"position_id": 1993, "txt": " You can use it as a playground to easily play around and patch any open-source project.", "x": "/html/body/div/div/div/article/p[41]/a[3]"}]}, "tags": [], "label": "content", "web_segment_id": 73, "global_sentence_id": 204, "edu_l1_label": "IOS"}, {"txt": " Here’s some example:", "language": "english", "position": {"atoms": [{"position_id": 1994, "txt": " Here’s some example:", "x": "/html/body/div/div/div/article/p[41]/a[3]"}]}, "tags": [], "label": "content", "web_segment_id": 73, "global_sentence_id": 205, "edu_l1_label": "IOS"}, {"txt": "# checkout libnvidia-container package source code locallynp checkout nixpkgs#libnvidia-container# modify the codevim checkout/src/cli/main.c# build the package with changes you made in the checkout folder and try it outnp build# output the patch for applying on the production environmentsnp patch > bugfix.patch# clean up the generated filesnp clean", "language": "english", "position": {"atoms": [{"position_id": 1996, "txt": "# checkout libnvidia-container package source code locally", "x": "/html/body/div/div/div/article/div[9]/div/pre/code/span[1]"}, {"position_id": 1998, "txt": "\nnp checkout nixpkgs#libnvidia-container\n\n", "x": "/html/body/div/div/div/article/div[9]/div/pre/code/span[1]"}, {"position_id": 2000, "txt": "# modify the code", "x": "/html/body/div/div/div/article/div[9]/div/pre/code/span[2]"}, {"position_id": 2002, "txt": "\nvim checkout/src/cli/main.c\n\n", "x": "/html/body/div/div/div/article/div[9]/div/pre/code/span[2]"}, {"position_id": 2004, "txt": "# build the package with changes you made in the checkout folder and try it out", "x": "/html/body/div/div/div/article/div[9]/div/pre/code/span[3]"}, {"position_id": 2006, "txt": "\nnp build\n\n", "x": "/html/body/div/div/div/article/div[9]/div/pre/code/span[3]"}, {"position_id": 2008, "txt": "# output the patch for applying on the production environments", "x": "/html/body/div/div/div/article/div[9]/div/pre/code/span[4]"}, {"position_id": 2010, "txt": "\nnp patch ", "x": "/html/body/div/div/div/article/div[9]/div/pre/code/span[4]"}, {"position_id": 2012, "txt": ">", "x": "/html/body/div/div/div/article/div[9]/div/pre/code/span[5]"}, {"position_id": 2014, "txt": " bugfix.patch\n\n", "x": "/html/body/div/div/div/article/div[9]/div/pre/code/span[5]"}, {"position_id": 2016, "txt": "# clean up the generated files", "x": "/html/body/div/div/div/article/div[9]/div/pre/code/span[6]"}, {"position_id": 2018, "txt": "\nnp clean\n", "x": "/html/body/div/div/div/article/div[9]/div/pre/code/span[6]"}]}, "tags": [], "label": "code", "web_segment_id": 74, "global_sentence_id": 206, "edu_l1_label": "EDU_O"}, {"txt": "I spent two days fixing some corner cases on this open-source project.", "language": "english", "position": {"atoms": [{"position_id": 2020, "txt": "I spent two days fixing some corner cases on this open-source project.", "x": "/html/body/div/div/div/article/p[42]"}]}, "tags": [], "label": "content", "web_segment_id": 75, "global_sentence_id": 207, "edu_l1_label": "IOS"}, {"txt": " It now works great!", "language": "english", "position": {"atoms": [{"position_id": 2021, "txt": " It now works great!", "x": "/html/body/div/div/div/article/p[42]"}]}, "tags": [], "label": "content", "web_segment_id": 75, "global_sentence_id": 208, "edu_l1_label": "IOS"}, {"txt": " I have the best shovel for digging the k8s-device-plugin crashing bug.", "language": "english", "position": {"atoms": [{"position_id": 2022, "txt": " I have the best shovel for digging the k8s-device-plugin crashing bug.", "x": "/html/body/div/div/div/article/p[42]"}]}, "tags": [], "label": "content", "web_segment_id": 75, "global_sentence_id": 209, "edu_l1_label": "IOS"}, {"txt": " It’s time to go back and dig the previous rabbit hole!", "language": "english", "position": {"atoms": [{"position_id": 2023, "txt": " It’s time to go back and dig the previous rabbit hole!", "x": "/html/body/div/div/div/article/p[42]"}]}, "tags": [], "label": "content", "web_segment_id": 75, "global_sentence_id": 210, "edu_l1_label": "IOS"}, {"txt": "Hunting the k8s-device-plugin crashing bug", "language": "english", "position": {"atoms": [{"position_id": 2025, "txt": "Hunting the k8s-device-plugin crashing bug", "x": "/html/body/div/div/div/article/h2[5]"}]}, "tags": ["h2"], "label": "title1", "web_segment_id": 76, "global_sentence_id": 211, "edu_l1_label": "BOS"}, {"txt": "One problem I faced when debugging the k8s-device-plugin crashing issue was that I didn’t know how it worked in detail.", "language": "english", "position": {"atoms": [{"position_id": 2027, "txt": "One problem I faced when debugging the k8s-device-plugin crashing issue was that I didn’t know how it worked in detail.", "x": "/html/body/div/div/div/article/p[43]"}]}, "tags": [], "label": "content", "web_segment_id": 77, "global_sentence_id": 212, "edu_l1_label": "IOS"}, {"txt": " So, I inserted many log writing lines into the code to see how it worked.", "language": "english", "position": {"atoms": [{"position_id": 2028, "txt": " So, I inserted many log writing lines into the code to see how it worked.", "x": "/html/body/div/div/div/article/p[43]"}]}, "tags": [], "label": "content", "web_segment_id": 77, "global_sentence_id": 213, "edu_l1_label": "IOS"}, {"txt": " Some may say hey, you can use a debugger.", "language": "english", "position": {"atoms": [{"position_id": 2029, "txt": " Some may say hey, you can use a debugger.", "x": "/html/body/div/div/div/article/p[43]"}]}, "tags": [], "label": "content", "web_segment_id": 77, "global_sentence_id": 214, "edu_l1_label": "IOS"}, {"txt": " Well, yeah, I know, but very often, it’s also very tedious to set up a debugger.", "language": "english", "position": {"atoms": [{"position_id": 2030, "txt": " Well, yeah, I know, but very often, it’s also very tedious to set up a debugger.", "x": "/html/body/div/div/div/article/p[43]"}]}, "tags": [], "label": "content", "web_segment_id": 77, "global_sentence_id": 215, "edu_l1_label": "IOS"}, {"txt": " I would use it if it comes in handy; otherwise, I would just insert logs.", "language": "english", "position": {"atoms": [{"position_id": 2031, "txt": " I would use it if it comes in handy; otherwise, I would just insert logs.", "x": "/html/body/div/div/div/article/p[43]"}]}, "tags": [], "label": "content", "web_segment_id": 77, "global_sentence_id": 216, "edu_l1_label": "IOS"}, {"txt": "I have been using Grok 3 for a while since its launch.", "language": "english", "position": {"atoms": [{"position_id": 2033, "txt": "I have been using ", "x": "/html/body/div/div/div/article/p[44]"}, {"position_id": 2035, "txt": "Grok 3", "x": "/html/body/div/div/div/article/p[44]/a[1]"}, {"position_id": 2037, "txt": " for a while since its launch.", "x": "/html/body/div/div/div/article/p[44]/a[1]"}]}, "tags": [], "label": "content", "web_segment_id": 78, "global_sentence_id": 217, "edu_l1_label": "IOS"}, {"txt": " I really enjoy it.", "language": "english", "position": {"atoms": [{"position_id": 2038, "txt": " I really enjoy it.", "x": "/html/body/div/div/div/article/p[44]/a[1]"}]}, "tags": [], "label": "content", "web_segment_id": 78, "global_sentence_id": 218, "edu_l1_label": "IOS"}, {"txt": " Sometimes, it provides made-up or wrong results but still very often provides sound analytics.", "language": "english", "position": {"atoms": [{"position_id": 2039, "txt": " Sometimes, it provides made-up or wrong results but still very often provides sound analytics.", "x": "/html/body/div/div/div/article/p[44]/a[1]"}]}, "tags": [], "label": "content", "web_segment_id": 78, "global_sentence_id": 219, "edu_l1_label": "IOS"}, {"txt": " I never intend to rely on it fully.", "language": "english", "position": {"atoms": [{"position_id": 2040, "txt": " I never intend to rely on it fully.", "x": "/html/body/div/div/div/article/p[44]/a[1]"}]}, "tags": [], "label": "content", "web_segment_id": 78, "global_sentence_id": 220, "edu_l1_label": "IOS"}, {"txt": " I take it as more like riding a bike.", "language": "english", "position": {"atoms": [{"position_id": 2041, "txt": " I take it as more like riding a bike.", "x": "/html/body/div/div/div/article/p[44]/a[1]"}]}, "tags": [], "label": "content", "web_segment_id": 78, "global_sentence_id": 221, "edu_l1_label": "IOS"}, {"txt": " I am still the one who rides the bike, not the bike riding me.", "language": "english", "position": {"atoms": [{"position_id": 2042, "txt": " I am still the one who rides the bike, not the bike riding me.", "x": "/html/body/div/div/div/article/p[44]/a[1]"}]}, "tags": [], "label": "content", "web_segment_id": 78, "global_sentence_id": 222, "edu_l1_label": "IOS"}, {"txt": " The code it generates still can’t always meet my standards.", "language": "english", "position": {"atoms": [{"position_id": 2043, "txt": " The code it generates still can’t always meet my standards.", "x": "/html/body/div/div/div/article/p[44]/a[1]"}]}, "tags": [], "label": "content", "web_segment_id": 78, "global_sentence_id": 223, "edu_l1_label": "IOS"}, {"txt": " When I jumped into debugging the Nvidia stuff, I quickly realized, well, despite that, I don’t trust the quality of the generated code from AI models, but if it’s throw-away code, it doesn’t matter.", "language": "english", "position": {"atoms": [{"position_id": 2044, "txt": " When I jumped into debugging the Nvidia stuff, I quickly realized, well, despite that, I don’t trust the quality of the generated code from AI models, but if it’s throw-away code, it doesn’t matter.", "x": "/html/body/div/div/div/article/p[44]/a[1]"}]}, "tags": [], "label": "content", "web_segment_id": 78, "global_sentence_id": 224, "edu_l1_label": "IOS"}, {"txt": " I don’t care about the quality as long as it works.", "language": "english", "position": {"atoms": [{"position_id": 2045, "txt": " I don’t care about the quality as long as it works.", "x": "/html/body/div/div/div/article/p[44]/a[1]"}]}, "tags": [], "label": "content", "web_segment_id": 78, "global_sentence_id": 225, "edu_l1_label": "IOS"}, {"txt": " Therefore, I asked Grok 3 to help me insert log entries into the source code I pasted from Nvidia’s GitHub repository, and it did a wonderful job.", "language": "english", "position": {"atoms": [{"position_id": 2046, "txt": " Therefore, I ", "x": "/html/body/div/div/div/article/p[44]/a[1]"}, {"position_id": 2048, "txt": "asked Grok 3 to help me insert log entries into the source code I pasted", "x": "/html/body/div/div/div/article/p[44]/a[2]"}, {"position_id": 2050, "txt": " from Nvidia’s GitHub repository, and it did a wonderful job.", "x": "/html/body/div/div/div/article/p[44]/a[2]"}]}, "tags": [], "label": "content", "web_segment_id": 78, "global_sentence_id": 226, "edu_l1_label": "IOS"}, {"txt": "Thanks to Grok 3, I no longer have to manually write log entries for debugging purposes and can instead focus on solving the actual problem.", "language": "english", "position": {"atoms": [{"position_id": 2052, "txt": "Thanks to Grok 3, I no longer have to manually write log entries for debugging purposes and can instead focus on solving the actual problem.", "x": "/html/body/div/div/div/article/p[45]"}]}, "tags": [], "label": "content", "web_segment_id": 79, "global_sentence_id": 227, "edu_l1_label": "IOS"}, {"txt": " Here’ss the log file generated from the update ldconfig hook patched by Grok 3.", "language": "english", "position": {"atoms": [{"position_id": 2053, "txt": " Here’ss the log file generated from the ", "x": "/html/body/div/div/div/article/p[45]"}, {"position_id": 2055, "txt": "update ldconfig hook", "x": "/html/body/div/div/div/article/p[45]/a"}, {"position_id": 2057, "txt": " patched by Grok 3.", "x": "/html/body/div/div/div/article/p[45]/a"}]}, "tags": [], "label": "content", "web_segment_id": 79, "global_sentence_id": 228, "edu_l1_label": "EDU_O"}, {"txt": "2025-02-26T17:", "language": "english", "position": {"atoms": [{"position_id": 2059, "txt": "2025-02-26T17:", "x": "/html/body/div/div/div/article/div[10]/div/pre/code"}]}, "tags": [], "label": "code", "web_segment_id": 80, "global_sentence_id": 229, "edu_l1_label": "EDU_O"}, {"txt": "23:", "language": "english", "position": {"atoms": [{"position_id": 2060, "txt": "23:", "x": "/html/body/div/div/div/article/div[10]/div/pre/code"}]}, "tags": [], "label": "code", "web_segment_id": 80, "global_sentence_id": 230, "edu_l1_label": "EDU_O"}, {"txt": "51-08:", "language": "english", "position": {"atoms": [{"position_id": 2061, "txt": "51-08:", "x": "/html/body/div/div/div/article/div[10]/div/pre/code"}]}, "tags": [], "label": "code", "web_segment_id": 80, "global_sentence_id": 231, "edu_l1_label": "EDU_O"}, {"txt": "00:", "language": "english", "position": {"atoms": [{"position_id": 2062, "txt": "00:", "x": "/html/body/div/div/div/article/div[10]/div/pre/code"}]}, "tags": [], "label": "code", "web_segment_id": 80, "global_sentence_id": 232, "edu_l1_label": "EDU_O"}, {"txt": " Starting update-ldcache2025-02-26T17:", "language": "english", "position": {"atoms": [{"position_id": 2063, "txt": " Starting update-ldcache\n2025-02-26T17:", "x": "/html/body/div/div/div/article/div[10]/div/pre/code"}]}, "tags": [], "label": "code", "web_segment_id": 80, "global_sentence_id": 233, "edu_l1_label": "EDU_O"}, {"txt": "23:", "language": "english", "position": {"atoms": [{"position_id": 2064, "txt": "23:", "x": "/html/body/div/div/div/article/div[10]/div/pre/code"}]}, "tags": [], "label": "code", "web_segment_id": 80, "global_sentence_id": 234, "edu_l1_label": "EDU_O"}, {"txt": "51-08:", "language": "english", "position": {"atoms": [{"position_id": 2065, "txt": "51-08:", "x": "/html/body/div/div/div/article/div[10]/div/pre/code"}]}, "tags": [], "label": "code", "web_segment_id": 80, "global_sentence_id": 235, "edu_l1_label": "EDU_O"}, {"txt": "00:", "language": "english", "position": {"atoms": [{"position_id": 2066, "txt": "00:", "x": "/html/body/div/div/div/article/div[10]/div/pre/code"}]}, "tags": [], "label": "code", "web_segment_id": 80, "global_sentence_id": 236, "edu_l1_label": "EDU_O"}, {"txt": " Loading container state from2025-02-26T17:", "language": "english", "position": {"atoms": [{"position_id": 2067, "txt": " Loading container state from\n2025-02-26T17:", "x": "/html/body/div/div/div/article/div[10]/div/pre/code"}]}, "tags": [], "label": "code", "web_segment_id": 80, "global_sentence_id": 237, "edu_l1_label": "EDU_O"}, {"txt": "23:", "language": "english", "position": {"atoms": [{"position_id": 2068, "txt": "23:", "x": "/html/body/div/div/div/article/div[10]/div/pre/code"}]}, "tags": [], "label": "code", "web_segment_id": 80, "global_sentence_id": 238, "edu_l1_label": "EDU_O"}, {"txt": "51-08:", "language": "english", "position": {"atoms": [{"position_id": 2069, "txt": "51-08:", "x": "/html/body/div/div/div/article/div[10]/div/pre/code"}]}, "tags": [], "label": "code", "web_segment_id": 80, "global_sentence_id": 239, "edu_l1_label": "EDU_O"}, {"txt": "00:", "language": "english", "position": {"atoms": [{"position_id": 2070, "txt": "00:", "x": "/html/body/div/div/div/article/div[10]/div/pre/code"}]}, "tags": [], "label": "code", "web_segment_id": 80, "global_sentence_id": 240, "edu_l1_label": "EDU_O"}, {"txt": " Loaded container state2025-02-26T17:", "language": "english", "position": {"atoms": [{"position_id": 2071, "txt": " Loaded container state\n2025-02-26T17:", "x": "/html/body/div/div/div/article/div[10]/div/pre/code"}]}, "tags": [], "label": "code", "web_segment_id": 80, "global_sentence_id": 241, "edu_l1_label": "EDU_O"}, {"txt": "23:", "language": "english", "position": {"atoms": [{"position_id": 2072, "txt": "23:", "x": "/html/body/div/div/div/article/div[10]/div/pre/code"}]}, "tags": [], "label": "code", "web_segment_id": 80, "global_sentence_id": 242, "edu_l1_label": "EDU_O"}, {"txt": "51-08:", "language": "english", "position": {"atoms": [{"position_id": 2073, "txt": "51-08:", "x": "/html/body/div/div/div/article/div[10]/div/pre/code"}]}, "tags": [], "label": "code", "web_segment_id": 80, "global_sentence_id": 243, "edu_l1_label": "EDU_O"}, {"txt": "00:", "language": "english", "position": {"atoms": [{"position_id": 2074, "txt": "00:", "x": "/html/body/div/div/div/article/div[10]/div/pre/code"}]}, "tags": [], "label": "code", "web_segment_id": 80, "global_sentence_id": 244, "edu_l1_label": "EDU_O"}, {"txt": " Getting container root2025-02-26T17:", "language": "english", "position": {"atoms": [{"position_id": 2075, "txt": " Getting container root\n2025-02-26T17:", "x": "/html/body/div/div/div/article/div[10]/div/pre/code"}]}, "tags": [], "label": "code", "web_segment_id": 80, "global_sentence_id": 245, "edu_l1_label": "EDU_O"}, {"txt": "23:", "language": "english", "position": {"atoms": [{"position_id": 2076, "txt": "23:", "x": "/html/body/div/div/div/article/div[10]/div/pre/code"}]}, "tags": [], "label": "code", "web_segment_id": 80, "global_sentence_id": 246, "edu_l1_label": "EDU_O"}, {"txt": "51-08:", "language": "english", "position": {"atoms": [{"position_id": 2077, "txt": "51-08:", "x": "/html/body/div/div/div/article/div[10]/div/pre/code"}]}, "tags": [], "label": "code", "web_segment_id": 80, "global_sentence_id": 247, "edu_l1_label": "EDU_O"}, {"txt": "00:", "language": "english", "position": {"atoms": [{"position_id": 2078, "txt": "00:", "x": "/html/body/div/div/div/article/div[10]/div/pre/code"}]}, "tags": [], "label": "code", "web_segment_id": 80, "global_sentence_id": 248, "edu_l1_label": "EDU_O"}, {"txt": " Determined container root:", "language": "english", "position": {"atoms": [{"position_id": 2079, "txt": " Determined container root:", "x": "/html/body/div/div/div/article/div[10]/div/pre/code"}]}, "tags": [], "label": "code", "web_segment_id": 80, "global_sentence_id": 249, "edu_l1_label": "EDU_O"}, {"txt": " /home/fangpen/.local/share/containers/storage/overlay/5c92b5c6053ce2643bcbe516adf268ecdf03244ab3a514ff2e539b6017dbcf0e/merged2025-02-26T17:", "language": "english", "position": {"atoms": [{"position_id": 2080, "txt": " /home/fangpen/.local/share/containers/storage/overlay/5c92b5c6053ce2643bcbe516adf268ecdf03244ab3a514ff2e539b6017dbcf0e/merged\n2025-02-26T17:", "x": "/html/body/div/div/div/article/div[10]/div/pre/code"}]}, "tags": [], "label": "code", "web_segment_id": 80, "global_sentence_id": 250, "edu_l1_label": "EDU_O"}, {"txt": "23:", "language": "english", "position": {"atoms": [{"position_id": 2081, "txt": "23:", "x": "/html/body/div/div/div/article/div[10]/div/pre/code"}]}, "tags": [], "label": "code", "web_segment_id": 80, "global_sentence_id": 251, "edu_l1_label": "EDU_O"}, {"txt": "51-08:", "language": "english", "position": {"atoms": [{"position_id": 2082, "txt": "51-08:", "x": "/html/body/div/div/div/article/div[10]/div/pre/code"}]}, "tags": [], "label": "code", "web_segment_id": 80, "global_sentence_id": 252, "edu_l1_label": "EDU_O"}, {"txt": "00:", "language": "english", "position": {"atoms": [{"position_id": 2083, "txt": "00:", "x": "/html/body/div/div/div/article/div[10]/div/pre/code"}]}, "tags": [], "label": "code", "web_segment_id": 80, "global_sentence_id": 253, "edu_l1_label": "EDU_O"}, {"txt": " Resolving ldconfig path from /nix/store/29mb4q8b5306f4gk2wh38h0c1akb0n97-glibc-2.40-36-bin/bin/ldconfig2025-02-26T17:", "language": "english", "position": {"atoms": [{"position_id": 2084, "txt": " Resolving ldconfig path from /nix/store/29mb4q8b5306f4gk2wh38h0c1akb0n97-glibc-2.40-36-bin/bin/ldconfig\n2025-02-26T17:", "x": "/html/body/div/div/div/article/div[10]/div/pre/code"}]}, "tags": [], "label": "code", "web_segment_id": 80, "global_sentence_id": 254, "edu_l1_label": "EDU_O"}, {"txt": "23:", "language": "english", "position": {"atoms": [{"position_id": 2085, "txt": "23:", "x": "/html/body/div/div/div/article/div[10]/div/pre/code"}]}, "tags": [], "label": "code", "web_segment_id": 80, "global_sentence_id": 255, "edu_l1_label": "EDU_O"}, {"txt": "51-08:", "language": "english", "position": {"atoms": [{"position_id": 2086, "txt": "51-08:", "x": "/html/body/div/div/div/article/div[10]/div/pre/code"}]}, "tags": [], "label": "code", "web_segment_id": 80, "global_sentence_id": 256, "edu_l1_label": "EDU_O"}, {"txt": "00:", "language": "english", "position": {"atoms": [{"position_id": 2087, "txt": "00:", "x": "/html/body/div/div/div/article/div[10]/div/pre/code"}]}, "tags": [], "label": "code", "web_segment_id": 80, "global_sentence_id": 257, "edu_l1_label": "EDU_O"}, {"txt": " Resolved ldconfig path:", "language": "english", "position": {"atoms": [{"position_id": 2088, "txt": " Resolved ldconfig path:", "x": "/html/body/div/div/div/article/div[10]/div/pre/code"}]}, "tags": [], "label": "code", "web_segment_id": 80, "global_sentence_id": 258, "edu_l1_label": "EDU_O"}, {"txt": " /nix/store/29mb4q8b5306f4gk2wh38h0c1akb0n97-glibc-2.40-36-bin/bin/ldconfig2025-02-26T17:", "language": "english", "position": {"atoms": [{"position_id": 2089, "txt": " /nix/store/29mb4q8b5306f4gk2wh38h0c1akb0n97-glibc-2.40-36-bin/bin/ldconfig\n2025-02-26T17:", "x": "/html/body/div/div/div/article/div[10]/div/pre/code"}]}, "tags": [], "label": "code", "web_segment_id": 80, "global_sentence_id": 259, "edu_l1_label": "EDU_O"}, {"txt": "23:", "language": "english", "position": {"atoms": [{"position_id": 2090, "txt": "23:", "x": "/html/body/div/div/div/article/div[10]/div/pre/code"}]}, "tags": [], "label": "code", "web_segment_id": 80, "global_sentence_id": 260, "edu_l1_label": "EDU_O"}, {"txt": "51-08:", "language": "english", "position": {"atoms": [{"position_id": 2091, "txt": "51-08:", "x": "/html/body/div/div/div/article/div[10]/div/pre/code"}]}, "tags": [], "label": "code", "web_segment_id": 80, "global_sentence_id": 261, "edu_l1_label": "EDU_O"}, {"txt": "00:", "language": "english", "position": {"atoms": [{"position_id": 2092, "txt": "00:", "x": "/html/body/div/div/div/article/div[10]/div/pre/code"}]}, "tags": [], "label": "code", "web_segment_id": 80, "global_sentence_id": 262, "edu_l1_label": "EDU_O"}, {"txt": " Checking for /etc/ld.so.cache in container2025-02-26T17:", "language": "english", "position": {"atoms": [{"position_id": 2093, "txt": " Checking for /etc/ld.so.cache in container\n2025-02-26T17:", "x": "/html/body/div/div/div/article/div[10]/div/pre/code"}]}, "tags": [], "label": "code", "web_segment_id": 80, "global_sentence_id": 263, "edu_l1_label": "EDU_O"}, {"txt": "23:", "language": "english", "position": {"atoms": [{"position_id": 2094, "txt": "23:", "x": "/html/body/div/div/div/article/div[10]/div/pre/code"}]}, "tags": [], "label": "code", "web_segment_id": 80, "global_sentence_id": 264, "edu_l1_label": "EDU_O"}, {"txt": "51-08:", "language": "english", "position": {"atoms": [{"position_id": 2095, "txt": "51-08:", "x": "/html/body/div/div/div/article/div[10]/div/pre/code"}]}, "tags": [], "label": "code", "web_segment_id": 80, "global_sentence_id": 265, "edu_l1_label": "EDU_O"}, {"txt": "00:", "language": "english", "position": {"atoms": [{"position_id": 2096, "txt": "00:", "x": "/html/body/div/div/div/article/div[10]/div/pre/code"}]}, "tags": [], "label": "code", "web_segment_id": 80, "global_sentence_id": 266, "edu_l1_label": "EDU_O"}, {"txt": " /etc/ld.so.cache exists, will update cache2025-02-26T17:", "language": "english", "position": {"atoms": [{"position_id": 2097, "txt": " /etc/ld.so.cache exists, will update cache\n2025-02-26T17:", "x": "/html/body/div/div/div/article/div[10]/div/pre/code"}]}, "tags": [], "label": "code", "web_segment_id": 80, "global_sentence_id": 267, "edu_l1_label": "EDU_O"}, {"txt": "23:", "language": "english", "position": {"atoms": [{"position_id": 2098, "txt": "23:", "x": "/html/body/div/div/div/article/div[10]/div/pre/code"}]}, "tags": [], "label": "code", "web_segment_id": 80, "global_sentence_id": 268, "edu_l1_label": "EDU_O"}, {"txt": "51-08:", "language": "english", "position": {"atoms": [{"position_id": 2099, "txt": "51-08:", "x": "/html/body/div/div/div/article/div[10]/div/pre/code"}]}, "tags": [], "label": "code", "web_segment_id": 80, "global_sentence_id": 269, "edu_l1_label": "EDU_O"}, {"txt": "00:", "language": "english", "position": {"atoms": [{"position_id": 2100, "txt": "00:", "x": "/html/body/div/div/div/article/div[10]/div/pre/code"}]}, "tags": [], "label": "code", "web_segment_id": 80, "global_sentence_id": 270, "edu_l1_label": "EDU_O"}, {"txt": " Creating config in /etc/ld.so.conf.d for folders:", "language": "english", "position": {"atoms": [{"position_id": 2101, "txt": " Creating config in /etc/ld.so.conf.d for folders:", "x": "/html/body/div/div/div/article/div[10]/div/pre/code"}]}, "tags": [], "label": "code", "web_segment_id": 80, "global_sentence_id": 271, "edu_l1_label": "EDU_O"}, {"txt": " /nix/store/x5522a7p46nnbwxjv8w942p6qps7x0lw-nvidia-x11-565.77-6.6.79/lib2025-02-26T17:", "language": "english", "position": {"atoms": [{"position_id": 2102, "txt": " /nix/store/x5522a7p46nnbwxjv8w942p6qps7x0lw-nvidia-x11-565.77-6.6.79/lib\n2025-02-26T17:", "x": "/html/body/div/div/div/article/div[10]/div/pre/code"}]}, "tags": [], "label": "code", "web_segment_id": 80, "global_sentence_id": 272, "edu_l1_label": "EDU_O"}, {"txt": "23:", "language": "english", "position": {"atoms": [{"position_id": 2103, "txt": "23:", "x": "/html/body/div/div/div/article/div[10]/div/pre/code"}]}, "tags": [], "label": "code", "web_segment_id": 80, "global_sentence_id": 273, "edu_l1_label": "EDU_O"}, {"txt": "51-08:", "language": "english", "position": {"atoms": [{"position_id": 2104, "txt": "51-08:", "x": "/html/body/div/div/div/article/div[10]/div/pre/code"}]}, "tags": [], "label": "code", "web_segment_id": 80, "global_sentence_id": 274, "edu_l1_label": "EDU_O"}, {"txt": "00:", "language": "english", "position": {"atoms": [{"position_id": 2105, "txt": "00:", "x": "/html/body/div/div/div/article/div[10]/div/pre/code"}]}, "tags": [], "label": "code", "web_segment_id": 80, "global_sentence_id": 275, "edu_l1_label": "EDU_O"}, {"txt": " Creating /etc/ld.so.conf.d if not exists2025-02-26T17:", "language": "english", "position": {"atoms": [{"position_id": 2106, "txt": " Creating /etc/ld.so.conf.d if not exists\n2025-02-26T17:", "x": "/html/body/div/div/div/article/div[10]/div/pre/code"}]}, "tags": [], "label": "code", "web_segment_id": 80, "global_sentence_id": 276, "edu_l1_label": "EDU_O"}, {"txt": "23:", "language": "english", "position": {"atoms": [{"position_id": 2107, "txt": "23:", "x": "/html/body/div/div/div/article/div[10]/div/pre/code"}]}, "tags": [], "label": "code", "web_segment_id": 80, "global_sentence_id": 277, "edu_l1_label": "EDU_O"}, {"txt": "51-08:", "language": "english", "position": {"atoms": [{"position_id": 2108, "txt": "51-08:", "x": "/html/body/div/div/div/article/div[10]/div/pre/code"}]}, "tags": [], "label": "code", "web_segment_id": 80, "global_sentence_id": 278, "edu_l1_label": "EDU_O"}, {"txt": "00:", "language": "english", "position": {"atoms": [{"position_id": 2109, "txt": "00:", "x": "/html/body/div/div/div/article/div[10]/div/pre/code"}]}, "tags": [], "label": "code", "web_segment_id": 80, "global_sentence_id": 279, "edu_l1_label": "EDU_O"}, {"txt": " Created /etc/ld.so.conf.d2025-02-26T17:", "language": "english", "position": {"atoms": [{"position_id": 2110, "txt": " Created /etc/ld.so.conf.d\n2025-02-26T17:", "x": "/html/body/div/div/div/article/div[10]/div/pre/code"}]}, "tags": [], "label": "code", "web_segment_id": 80, "global_sentence_id": 280, "edu_l1_label": "EDU_O"}, {"txt": "23:", "language": "english", "position": {"atoms": [{"position_id": 2111, "txt": "23:", "x": "/html/body/div/div/div/article/div[10]/div/pre/code"}]}, "tags": [], "label": "code", "web_segment_id": 80, "global_sentence_id": 281, "edu_l1_label": "EDU_O"}, {"txt": "51-08:", "language": "english", "position": {"atoms": [{"position_id": 2112, "txt": "51-08:", "x": "/html/body/div/div/div/article/div[10]/div/pre/code"}]}, "tags": [], "label": "code", "web_segment_id": 80, "global_sentence_id": 282, "edu_l1_label": "EDU_O"}, {"txt": "00:", "language": "english", "position": {"atoms": [{"position_id": 2113, "txt": "00:", "x": "/html/body/div/div/div/article/div[10]/div/pre/code"}]}, "tags": [], "label": "code", "web_segment_id": 80, "global_sentence_id": 283, "edu_l1_label": "EDU_O"}, {"txt": " Creating temp config file in /etc/ld.so.conf.d2025-02-26T17:", "language": "english", "position": {"atoms": [{"position_id": 2114, "txt": " Creating temp config file in /etc/ld.so.conf.d\n2025-02-26T17:", "x": "/html/body/div/div/div/article/div[10]/div/pre/code"}]}, "tags": [], "label": "code", "web_segment_id": 80, "global_sentence_id": 284, "edu_l1_label": "EDU_O"}, {"txt": "23:", "language": "english", "position": {"atoms": [{"position_id": 2115, "txt": "23:", "x": "/html/body/div/div/div/article/div[10]/div/pre/code"}]}, "tags": [], "label": "code", "web_segment_id": 80, "global_sentence_id": 285, "edu_l1_label": "EDU_O"}, {"txt": "51-08:", "language": "english", "position": {"atoms": [{"position_id": 2116, "txt": "51-08:", "x": "/html/body/div/div/div/article/div[10]/div/pre/code"}]}, "tags": [], "label": "code", "web_segment_id": 80, "global_sentence_id": 286, "edu_l1_label": "EDU_O"}, {"txt": "00:", "language": "english", "position": {"atoms": [{"position_id": 2117, "txt": "00:", "x": "/html/body/div/div/div/article/div[10]/div/pre/code"}]}, "tags": [], "label": "code", "web_segment_id": 80, "global_sentence_id": 287, "edu_l1_label": "EDU_O"}, {"txt": " Created temp config file:", "language": "english", "position": {"atoms": [{"position_id": 2118, "txt": " Created temp config file:", "x": "/html/body/div/div/div/article/div[10]/div/pre/code"}]}, "tags": [], "label": "code", "web_segment_id": 80, "global_sentence_id": 288, "edu_l1_label": "EDU_O"}, {"txt": " /home/fangpen/.local/share/containers/storage/overlay/5c92b5c6053ce2643bcbe516adf268ecdf03244ab3a514ff2e539b6017dbcf0e/merged/etc/ld.so.conf.d/nvcr-4265966838.conf2025-02-26T17:", "language": "english", "position": {"atoms": [{"position_id": 2119, "txt": " /home/fangpen/.local/share/containers/storage/overlay/5c92b5c6053ce2643bcbe516adf268ecdf03244ab3a514ff2e539b6017dbcf0e/merged/etc/ld.so.conf.d/nvcr-4265966838.conf\n2025-02-26T17:", "x": "/html/body/div/div/div/article/div[10]/div/pre/code"}]}, "tags": [], "label": "code", "web_segment_id": 80, "global_sentence_id": 289, "edu_l1_label": "EDU_O"}, {"txt": "23:", "language": "english", "position": {"atoms": [{"position_id": 2120, "txt": "23:", "x": "/html/body/div/div/div/article/div[10]/div/pre/code"}]}, "tags": [], "label": "code", "web_segment_id": 80, "global_sentence_id": 290, "edu_l1_label": "EDU_O"}, {"txt": "51-08:", "language": "english", "position": {"atoms": [{"position_id": 2121, "txt": "51-08:", "x": "/html/body/div/div/div/article/div[10]/div/pre/code"}]}, "tags": [], "label": "code", "web_segment_id": 80, "global_sentence_id": 291, "edu_l1_label": "EDU_O"}, {"txt": "00:", "language": "english", "position": {"atoms": [{"position_id": 2122, "txt": "00:", "x": "/html/body/div/div/div/article/div[10]/div/pre/code"}]}, "tags": [], "label": "code", "web_segment_id": 80, "global_sentence_id": 292, "edu_l1_label": "EDU_O"}, {"txt": " Writing folders to config file:", "language": "english", "position": {"atoms": [{"position_id": 2123, "txt": " Writing folders to config file:", "x": "/html/body/div/div/div/article/div[10]/div/pre/code"}]}, "tags": [], "label": "code", "web_segment_id": 80, "global_sentence_id": 293, "edu_l1_label": "EDU_O"}, {"txt": " /nix/store/x5522a7p46nnbwxjv8w942p6qps7x0lw-nvidia-x11-565.77-6.6.79/lib2025-02-26T17:", "language": "english", "position": {"atoms": [{"position_id": 2124, "txt": " /nix/store/x5522a7p46nnbwxjv8w942p6qps7x0lw-nvidia-x11-565.77-6.6.79/lib\n2025-02-26T17:", "x": "/html/body/div/div/div/article/div[10]/div/pre/code"}]}, "tags": [], "label": "code", "web_segment_id": 80, "global_sentence_id": 294, "edu_l1_label": "EDU_O"}, {"txt": "23:", "language": "english", "position": {"atoms": [{"position_id": 2125, "txt": "23:", "x": "/html/body/div/div/div/article/div[10]/div/pre/code"}]}, "tags": [], "label": "code", "web_segment_id": 80, "global_sentence_id": 295, "edu_l1_label": "EDU_O"}, {"txt": "51-08:", "language": "english", "position": {"atoms": [{"position_id": 2126, "txt": "51-08:", "x": "/html/body/div/div/div/article/div[10]/div/pre/code"}]}, "tags": [], "label": "code", "web_segment_id": 80, "global_sentence_id": 296, "edu_l1_label": "EDU_O"}, {"txt": "00:", "language": "english", "position": {"atoms": [{"position_id": 2127, "txt": "00:", "x": "/html/body/div/div/div/article/div[10]/div/pre/code"}]}, "tags": [], "label": "code", "web_segment_id": 80, "global_sentence_id": 297, "edu_l1_label": "EDU_O"}, {"txt": " Added folder:", "language": "english", "position": {"atoms": [{"position_id": 2128, "txt": " Added folder:", "x": "/html/body/div/div/div/article/div[10]/div/pre/code"}]}, "tags": [], "label": "code", "web_segment_id": 80, "global_sentence_id": 298, "edu_l1_label": "EDU_O"}, {"txt": " /nix/store/x5522a7p46nnbwxjv8w942p6qps7x0lw-nvidia-x11-565.77-6.6.79/lib2025-02-26T17:", "language": "english", "position": {"atoms": [{"position_id": 2129, "txt": " /nix/store/x5522a7p46nnbwxjv8w942p6qps7x0lw-nvidia-x11-565.77-6.6.79/lib\n2025-02-26T17:", "x": "/html/body/div/div/div/article/div[10]/div/pre/code"}]}, "tags": [], "label": "code", "web_segment_id": 80, "global_sentence_id": 299, "edu_l1_label": "EDU_O"}, {"txt": "23:", "language": "english", "position": {"atoms": [{"position_id": 2130, "txt": "23:", "x": "/html/body/div/div/div/article/div[10]/div/pre/code"}]}, "tags": [], "label": "code", "web_segment_id": 80, "global_sentence_id": 300, "edu_l1_label": "EDU_O"}, {"txt": "51-08:", "language": "english", "position": {"atoms": [{"position_id": 2131, "txt": "51-08:", "x": "/html/body/div/div/div/article/div[10]/div/pre/code"}]}, "tags": [], "label": "code", "web_segment_id": 80, "global_sentence_id": 301, "edu_l1_label": "EDU_O"}, {"txt": "00:", "language": "english", "position": {"atoms": [{"position_id": 2132, "txt": "00:", "x": "/html/body/div/div/div/article/div[10]/div/pre/code"}]}, "tags": [], "label": "code", "web_segment_id": 80, "global_sentence_id": 302, "edu_l1_label": "EDU_O"}, {"txt": " Setting permissions on config file to 06442025-02-26T17:", "language": "english", "position": {"atoms": [{"position_id": 2133, "txt": " Setting permissions on config file to 0644\n2025-02-26T17:", "x": "/html/body/div/div/div/article/div[10]/div/pre/code"}]}, "tags": [], "label": "code", "web_segment_id": 80, "global_sentence_id": 303, "edu_l1_label": "EDU_O"}, {"txt": "23:", "language": "english", "position": {"atoms": [{"position_id": 2134, "txt": "23:", "x": "/html/body/div/div/div/article/div[10]/div/pre/code"}]}, "tags": [], "label": "code", "web_segment_id": 80, "global_sentence_id": 304, "edu_l1_label": "EDU_O"}, {"txt": "51-08:", "language": "english", "position": {"atoms": [{"position_id": 2135, "txt": "51-08:", "x": "/html/body/div/div/div/article/div[10]/div/pre/code"}]}, "tags": [], "label": "code", "web_segment_id": 80, "global_sentence_id": 305, "edu_l1_label": "EDU_O"}, {"txt": "00:", "language": "english", "position": {"atoms": [{"position_id": 2136, "txt": "00:", "x": "/html/body/div/div/div/article/div[10]/div/pre/code"}]}, "tags": [], "label": "code", "web_segment_id": 80, "global_sentence_id": 306, "edu_l1_label": "EDU_O"}, {"txt": " Set permissions on config file2025-02-26T17:", "language": "english", "position": {"atoms": [{"position_id": 2137, "txt": " Set permissions on config file\n2025-02-26T17:", "x": "/html/body/div/div/div/article/div[10]/div/pre/code"}]}, "tags": [], "label": "code", "web_segment_id": 80, "global_sentence_id": 307, "edu_l1_label": "EDU_O"}, {"txt": "23:", "language": "english", "position": {"atoms": [{"position_id": 2138, "txt": "23:", "x": "/html/body/div/div/div/article/div[10]/div/pre/code"}]}, "tags": [], "label": "code", "web_segment_id": 80, "global_sentence_id": 308, "edu_l1_label": "EDU_O"}, {"txt": "51-08:", "language": "english", "position": {"atoms": [{"position_id": 2139, "txt": "51-08:", "x": "/html/body/div/div/div/article/div[10]/div/pre/code"}]}, "tags": [], "label": "code", "web_segment_id": 80, "global_sentence_id": 309, "edu_l1_label": "EDU_O"}, {"txt": "00:", "language": "english", "position": {"atoms": [{"position_id": 2140, "txt": "00:", "x": "/html/body/div/div/div/article/div[10]/div/pre/code"}]}, "tags": [], "label": "code", "web_segment_id": 80, "global_sentence_id": 310, "edu_l1_label": "EDU_O"}, {"txt": " Preparing ldconfig arguments:", "language": "english", "position": {"atoms": [{"position_id": 2141, "txt": " Preparing ldconfig arguments:", "x": "/html/body/div/div/div/article/div[10]/div/pre/code"}]}, "tags": [], "label": "code", "web_segment_id": 80, "global_sentence_id": 311, "edu_l1_label": "EDU_O"}, {"txt": " ldconfig -r /home/fangpen/.local/share/containers/storage/overlay/5c92b5c6053ce2643bcbe516adf268ecdf03244ab3a514ff2e539b6017dbcf0e/merged -C /etc/ld.so.cache -f /etc/ld.so.conf2025-02-26T17:", "language": "english", "position": {"atoms": [{"position_id": 2142, "txt": " ldconfig -r /home/fangpen/.local/share/containers/storage/overlay/5c92b5c6053ce2643bcbe516adf268ecdf03244ab3a514ff2e539b6017dbcf0e/merged -C /etc/ld.so.cache -f /etc/ld.so.conf\n2025-02-26T17:", "x": "/html/body/div/div/div/article/div[10]/div/pre/code"}]}, "tags": [], "label": "code", "web_segment_id": 80, "global_sentence_id": 312, "edu_l1_label": "EDU_O"}, {"txt": "23:", "language": "english", "position": {"atoms": [{"position_id": 2143, "txt": "23:", "x": "/html/body/div/div/div/article/div[10]/div/pre/code"}]}, "tags": [], "label": "code", "web_segment_id": 80, "global_sentence_id": 313, "edu_l1_label": "EDU_O"}, {"txt": "51-08:", "language": "english", "position": {"atoms": [{"position_id": 2144, "txt": "51-08:", "x": "/html/body/div/div/div/article/div[10]/div/pre/code"}]}, "tags": [], "label": "code", "web_segment_id": 80, "global_sentence_id": 314, "edu_l1_label": "EDU_O"}, {"txt": "00:", "language": "english", "position": {"atoms": [{"position_id": 2145, "txt": "00:", "x": "/html/body/div/div/div/article/div[10]/div/pre/code"}]}, "tags": [], "label": "code", "web_segment_id": 80, "global_sentence_id": 315, "edu_l1_label": "EDU_O"}, {"txt": " Executing ldconfig with args:", "language": "english", "position": {"atoms": [{"position_id": 2146, "txt": " Executing ldconfig with args:", "x": "/html/body/div/div/div/article/div[10]/div/pre/code"}]}, "tags": [], "label": "code", "web_segment_id": 80, "global_sentence_id": 316, "edu_l1_label": "EDU_O"}, {"txt": " ldconfig -r /home/fangpen/.local/share/containers/storage/overlay/5c92b5c6053ce2643bcbe516adf268ecdf03244ab3a514ff2e539b6017dbcf0e/merged -C /etc/ld.so.cache -f /etc/ld.so.conf", "language": "english", "position": {"atoms": [{"position_id": 2147, "txt": " ldconfig -r /home/fangpen/.local/share/containers/storage/overlay/5c92b5c6053ce2643bcbe516adf268ecdf03244ab3a514ff2e539b6017dbcf0e/merged -C /etc/ld.so.cache -f /etc/ld.so.conf\n", "x": "/html/body/div/div/div/article/div[10]/div/pre/code"}]}, "tags": [], "label": "code", "web_segment_id": 80, "global_sentence_id": 317, "edu_l1_label": "EDU_O"}, {"txt": "With the logs, I tried things quickly and understood the flow of the code.", "language": "english", "position": {"atoms": [{"position_id": 2149, "txt": "With the logs, I tried things quickly and understood the flow of the code.", "x": "/html/body/div/div/div/article/p[46]"}]}, "tags": [], "label": "content", "web_segment_id": 81, "global_sentence_id": 318, "edu_l1_label": "IOS"}, {"txt": " However, since the problem happens inside the container namespaces, I cannot view them from my non-namespace environment.", "language": "english", "position": {"atoms": [{"position_id": 2150, "txt": " However, since the problem happens inside the container namespaces, I cannot view them from my non-namespace environment.", "x": "/html/body/div/div/div/article/p[46]"}]}, "tags": [], "label": "content", "web_segment_id": 81, "global_sentence_id": 319, "edu_l1_label": "IOS"}, {"txt": " I need to get a shell inside the namespace to better understand what’s going on.", "language": "english", "position": {"atoms": [{"position_id": 2151, "txt": " I need to get a shell inside the namespace to better understand what’s going on.", "x": "/html/body/div/div/div/article/p[46]"}]}, "tags": [], "label": "content", "web_segment_id": 81, "global_sentence_id": 320, "edu_l1_label": "IOS"}, {"txt": " So, after I narrowed down the problem to the update ldconfig hook, I added a long sleep before the Nvidia container runtime makes the exec call to ldconfig so that I have time to look into it.", "language": "english", "position": {"atoms": [{"position_id": 2152, "txt": " So, after I narrowed down the problem to the update ldconfig hook, I added a long sleep before the Nvidia container runtime makes the exec call to ldconfig so that I have time to look into it.", "x": "/html/body/div/div/div/article/p[46]"}]}, "tags": [], "label": "content", "web_segment_id": 81, "global_sentence_id": 321, "edu_l1_label": "IOS"}, {"txt": "The createContainer hook configuration at /var/run/cdi/nvidia-container-toolkit.json:", "language": "english", "position": {"atoms": [{"position_id": 2154, "txt": "The ", "x": "/html/body/div/div/div/article/p[47]"}, {"position_id": 2156, "txt": "createContainer", "x": "/html/body/div/div/div/article/p[47]/code[1]"}, {"position_id": 2158, "txt": " hook configuration at ", "x": "/html/body/div/div/div/article/p[47]/code[1]"}, {"position_id": 2160, "txt": "/var/run/cdi/nvidia-container-toolkit.json", "x": "/html/body/div/div/div/article/p[47]/code[2]"}, {"position_id": 2162, "txt": ":", "x": "/html/body/div/div/div/article/p[47]/code[2]"}]}, "tags": [], "label": "content", "web_segment_id": 82, "global_sentence_id": 322, "edu_l1_label": "IOS"}, {"txt": "{\"_\":\"", "language": "english", "position": {"atoms": [{"position_id": 2164, "txt": "{", "x": "/html/body/div/div/div/article/div[11]/div/pre/code/span[1]"}, {"position_id": 2166, "txt": "\"_\"", "x": "/html/body/div/div/div/article/div[11]/div/pre/code/span[3]"}, {"position_id": 2168, "txt": ":", "x": "/html/body/div/div/div/article/div[11]/div/pre/code/span[4]"}, {"position_id": 2170, "txt": "\"", "x": "/html/body/div/div/div/article/div[11]/div/pre/code/span[6]"}]}, "tags": [], "label": "code", "web_segment_id": 83, "global_sentence_id": 323, "edu_l1_label": "EDU_O"}, {"txt": "// ...\",\"hooks\":", "language": "english", "position": {"atoms": [{"position_id": 2171, "txt": "// ...\"", "x": "/html/body/div/div/div/article/div[11]/div/pre/code/span[6]"}, {"position_id": 2173, "txt": ",", "x": "/html/body/div/div/div/article/div[11]/div/pre/code/span[7]"}, {"position_id": 2175, "txt": "\"hooks\"", "x": "/html/body/div/div/div/article/div[11]/div/pre/code/span[9]"}, {"position_id": 2177, "txt": ":", "x": "/html/body/div/div/div/article/div[11]/div/pre/code/span[10]"}]}, "tags": [], "label": "code", "web_segment_id": 83, "global_sentence_id": 324, "edu_l1_label": "EDU_O"}, {"txt": "[{\"hookName\":\"", "language": "english", "position": {"atoms": [{"position_id": 2179, "txt": "[", "x": "/html/body/div/div/div/article/div[11]/div/pre/code/span[12]"}, {"position_id": 2181, "txt": "{", "x": "/html/body/div/div/div/article/div[11]/div/pre/code/span[14]"}, {"position_id": 2183, "txt": "\"hookName\"", "x": "/html/body/div/div/div/article/div[11]/div/pre/code/span[16]"}, {"position_id": 2185, "txt": ":", "x": "/html/body/div/div/div/article/div[11]/div/pre/code/span[17]"}, {"position_id": 2187, "txt": "\"", "x": "/html/body/div/div/div/article/div[11]/div/pre/code/span[19]"}]}, "tags": [], "label": "code", "web_segment_id": 83, "global_sentence_id": 325, "edu_l1_label": "EDU_O"}, {"txt": "createContainer\",\"path\":\"", "language": "english", "position": {"atoms": [{"position_id": 2188, "txt": "createContainer\"", "x": "/html/body/div/div/div/article/div[11]/div/pre/code/span[19]"}, {"position_id": 2190, "txt": ",", "x": "/html/body/div/div/div/article/div[11]/div/pre/code/span[20]"}, {"position_id": 2192, "txt": "\"path\"", "x": "/html/body/div/div/div/article/div[11]/div/pre/code/span[22]"}, {"position_id": 2194, "txt": ":", "x": "/html/body/div/div/div/article/div[11]/div/pre/code/span[23]"}, {"position_id": 2196, "txt": "\"", "x": "/html/body/div/div/div/article/div[11]/div/pre/code/span[25]"}]}, "tags": [], "label": "code", "web_segment_id": 83, "global_sentence_id": 326, "edu_l1_label": "EDU_O"}, {"txt": "//nix/store/il5kz2p67hdd05c9gmg8m5c5l8gbrk90-container-toolkit-container-toolkit-1.15.0-rc.3/bin/nvidia-ctk\",\"args\":", "language": "english", "position": {"atoms": [{"position_id": 2197, "txt": "//nix/store/il5kz2p67hdd05c9gmg8m5c5l8gbrk90-container-toolkit-container-toolkit-1.15.0-rc.3/bin/nvidia-ctk\"", "x": "/html/body/div/div/div/article/div[11]/div/pre/code/span[25]"}, {"position_id": 2199, "txt": ",", "x": "/html/body/div/div/div/article/div[11]/div/pre/code/span[26]"}, {"position_id": 2201, "txt": "\"args\"", "x": "/html/body/div/div/div/article/div[11]/div/pre/code/span[28]"}, {"position_id": 2203, "txt": ":", "x": "/html/body/div/div/div/article/div[11]/div/pre/code/span[29]"}]}, "tags": [], "label": "code", "web_segment_id": 83, "global_sentence_id": 327, "edu_l1_label": "EDU_O"}, {"txt": "[\"nvidia-ctk\",\"hook\",\"update-ldcache\",\"--ldconfig-path\",\"/nix/store/29mb4q8b5306f4gk2wh38h0c1akb0n97-glibc-2.40-36-bin/bin/ldconfig\",\"--folder\",\"/nix/store/x5522a7p46nnbwxjv8w942p6qps7x0lw-nvidia-x11-565.77-6.6.79/lib\"", "language": "english", "position": {"atoms": [{"position_id": 2205, "txt": "[", "x": "/html/body/div/div/div/article/div[11]/div/pre/code/span[31]"}, {"position_id": 2207, "txt": "\"nvidia-ctk\"", "x": "/html/body/div/div/div/article/div[11]/div/pre/code/span[33]"}, {"position_id": 2209, "txt": ",", "x": "/html/body/div/div/div/article/div[11]/div/pre/code/span[34]"}, {"position_id": 2211, "txt": "\"hook\"", "x": "/html/body/div/div/div/article/div[11]/div/pre/code/span[36]"}, {"position_id": 2213, "txt": ",", "x": "/html/body/div/div/div/article/div[11]/div/pre/code/span[37]"}, {"position_id": 2215, "txt": "\"update-ldcache\"", "x": "/html/body/div/div/div/article/div[11]/div/pre/code/span[39]"}, {"position_id": 2217, "txt": ",", "x": "/html/body/div/div/div/article/div[11]/div/pre/code/span[40]"}, {"position_id": 2219, "txt": "\"--ldconfig-path\"", "x": "/html/body/div/div/div/article/div[11]/div/pre/code/span[42]"}, {"position_id": 2221, "txt": ",", "x": "/html/body/div/div/div/article/div[11]/div/pre/code/span[43]"}, {"position_id": 2223, "txt": "\"/nix/store/29mb4q8b5306f4gk2wh38h0c1akb0n97-glibc-2.40-36-bin/bin/ldconfig\"", "x": "/html/body/div/div/div/article/div[11]/div/pre/code/span[45]"}, {"position_id": 2225, "txt": ",", "x": "/html/body/div/div/div/article/div[11]/div/pre/code/span[46]"}, {"position_id": 2227, "txt": "\"--folder\"", "x": "/html/body/div/div/div/article/div[11]/div/pre/code/span[48]"}, {"position_id": 2229, "txt": ",", "x": "/html/body/div/div/div/article/div[11]/div/pre/code/span[49]"}, {"position_id": 2231, "txt": "\"/nix/store/x5522a7p46nnbwxjv8w942p6qps7x0lw-nvidia-x11-565.77-6.6.79/lib\"", "x": "/html/body/div/div/div/article/div[11]/div/pre/code/span[51]"}]}, "tags": [], "label": "code", "web_segment_id": 83, "global_sentence_id": 328, "edu_l1_label": "EDU_O"}, {"txt": "]}]}", "language": "english", "position": {"atoms": [{"position_id": 2233, "txt": "]", "x": "/html/body/div/div/div/article/div[11]/div/pre/code/span[53]"}, {"position_id": 2235, "txt": "}", "x": "/html/body/div/div/div/article/div[11]/div/pre/code/span[55]"}, {"position_id": 2237, "txt": "]", "x": "/html/body/div/div/div/article/div[11]/div/pre/code/span[57]"}, {"position_id": 2239, "txt": "}", "x": "/html/body/div/div/div/article/div[11]/div/pre/code/span[59]"}]}, "tags": [], "label": "code", "web_segment_id": 83, "global_sentence_id": 329, "edu_l1_label": "EDU_O"}, {"txt": "After the hook sleeps and hangs, I ran", "language": "english", "position": {"atoms": [{"position_id": 2241, "txt": "After the hook sleeps and hangs, I ran", "x": "/html/body/div/div/div/article/p[48]"}]}, "tags": [], "label": "content", "web_segment_id": 84, "global_sentence_id": 330, "edu_l1_label": "IOS"}, {"txt": "$ lsns        NS TYPE   NPROCS    PID USER    COMMAND( ... omit ...)4026533991 mnt         1 388163 fangpen /bin/sh4026533992 uts         1 388163 fangpen /bin/sh4026533993 ipc         1 388163 fangpen /bin/sh4026533994 pid         1 388163 fangpen /bin/sh4026533995 cgroup      1 388163 fangpen /bin/sh", "language": "english", "position": {"atoms": [{"position_id": 2243, "txt": "$ ", "x": "/html/body/div/div/div/article/div[12]/div/pre/code/span[1]"}, {"position_id": 2245, "txt": "lsns\n        NS TYPE   NPROCS    PID USER    COMMAND\n", "x": "/html/body/div/div/div/article/div[12]/div/pre/code/span[1]"}, {"position_id": 2247, "txt": "(", "x": "/html/body/div/div/div/article/div[12]/div/pre/code/span[2]"}, {"position_id": 2249, "txt": " ... omit ...", "x": "/html/body/div/div/div/article/div[12]/div/pre/code/span[2]"}, {"position_id": 2251, "txt": ")", "x": "/html/body/div/div/div/article/div[12]/div/pre/code/span[3]"}, {"position_id": 2253, "txt": "\n4026533991 mnt         1 388163 fangpen /bin/sh\n4026533992 uts         1 388163 fangpen /bin/sh\n4026533993 ipc         1 388163 fangpen /bin/sh\n4026533994 pid         1 388163 fangpen /bin/sh\n4026533995 cgroup      1 388163 fangpen /bin/sh\n", "x": "/html/body/div/div/div/article/div[12]/div/pre/code/span[3]"}]}, "tags": [], "label": "code", "web_segment_id": 85, "global_sentence_id": 331, "edu_l1_label": "EDU_O"}, {"txt": "To find out the pid of the container process, I just launched via podman to reproduce the bug.", "language": "english", "position": {"atoms": [{"position_id": 2255, "txt": "To find out the pid of the container process, I just launched via podman to reproduce the bug.", "x": "/html/body/div/div/div/article/p[49]"}]}, "tags": [], "label": "content", "web_segment_id": 86, "global_sentence_id": 332, "edu_l1_label": "IOS"}, {"txt": " Then, I run this to attach a shell into the namespace:", "language": "english", "position": {"atoms": [{"position_id": 2256, "txt": " Then, I run this to attach a shell into the namespace:", "x": "/html/body/div/div/div/article/p[49]"}]}, "tags": [], "label": "content", "web_segment_id": 86, "global_sentence_id": 333, "edu_l1_label": "IOS"}, {"txt": "sudo enterns -t <CONTAINER_PID> -a /bin/sh", "language": "english", "position": {"atoms": [{"position_id": 2258, "txt": "sudo ", "x": "/html/body/div/div/div/article/div[13]/div/pre/code/span[1]"}, {"position_id": 2260, "txt": "enterns ", "x": "/html/body/div/div/div/article/div[13]/div/pre/code/span[1]"}, {"position_id": 2262, "txt": "-t", "x": "/html/body/div/div/div/article/div[13]/div/pre/code/span[2]"}, {"position_id": 2264, "txt": " <CONTAINER_PID> ", "x": "/html/body/div/div/div/article/div[13]/div/pre/code/span[2]"}, {"position_id": 2266, "txt": "-a", "x": "/html/body/div/div/div/article/div[13]/div/pre/code/span[3]"}, {"position_id": 2268, "txt": " /bin/sh\n", "x": "/html/body/div/div/div/article/div[13]/div/pre/code/span[3]"}]}, "tags": [], "label": "code", "web_segment_id": 87, "global_sentence_id": 334, "edu_l1_label": "EDU_O"}, {"txt": "Next, I ran the ldconfig command, which is supposed to run with the -p argument to see the current ldconfig cache.", "language": "english", "position": {"atoms": [{"position_id": 2270, "txt": "Next, I ran the ldconfig command, which is supposed to run with the ", "x": "/html/body/div/div/div/article/p[50]"}, {"position_id": 2272, "txt": "-p", "x": "/html/body/div/div/div/article/p[50]/code"}, {"position_id": 2274, "txt": " argument to see the current ldconfig cache.", "x": "/html/body/div/div/div/article/p[50]/code"}]}, "tags": [], "label": "content", "web_segment_id": 88, "global_sentence_id": 335, "edu_l1_label": "IOS"}, {"txt": "$ ldconfig -p131 libs found in cache `/etc/ld.so.cache'        ( ... omit ...)        libc.so.6 (libc6,x86-64, OS ABI:", "language": "english", "position": {"atoms": [{"position_id": 2276, "txt": "$ ldconfig -p\n131 libs found in cache `/etc/ld.so.cache'\n        ( ... omit ...)\n        libc.so.6 (libc6,x86-64, OS ABI:", "x": "/html/body/div/div/div/article/div[14]/div/pre/code"}]}, "tags": [], "label": "code", "web_segment_id": 89, "global_sentence_id": 336, "edu_l1_label": "EDU_O"}, {"txt": " Linux 3.2.0) => /lib64/libc.so.6        ( ... omit ...)", "language": "english", "position": {"atoms": [{"position_id": 2277, "txt": " Linux 3.2.0) => /lib64/libc.so.6\n        ( ... omit ...)", "x": "/html/body/div/div/div/article/div[14]/div/pre/code"}]}, "tags": [], "label": "code", "web_segment_id": 89, "global_sentence_id": 337, "edu_l1_label": "EDU_O"}, {"txt": "Cache generated by:", "language": "english", "position": {"atoms": [{"position_id": 2278, "txt": "\nCache generated by:", "x": "/html/body/div/div/div/article/div[14]/div/pre/code"}]}, "tags": [], "label": "code", "web_segment_id": 89, "global_sentence_id": 338, "edu_l1_label": "EDU_O"}, {"txt": " ldconfig (GNU libc) stable release version 2.34", "language": "english", "position": {"atoms": [{"position_id": 2279, "txt": " ldconfig (GNU libc) stable release version 2.34\n", "x": "/html/body/div/div/div/article/div[14]/div/pre/code"}]}, "tags": [], "label": "code", "web_segment_id": 89, "global_sentence_id": 339, "edu_l1_label": "EDU_O"}, {"txt": "Then, run the ldconfig without -p again to update cache and then with -p again to see the current cache:", "language": "english", "position": {"atoms": [{"position_id": 2281, "txt": "Then, run the ldconfig without ", "x": "/html/body/div/div/div/article/p[51]"}, {"position_id": 2283, "txt": "-p", "x": "/html/body/div/div/div/article/p[51]/code[1]"}, {"position_id": 2285, "txt": " again to update cache and then with ", "x": "/html/body/div/div/div/article/p[51]/code[1]"}, {"position_id": 2287, "txt": "-p", "x": "/html/body/div/div/div/article/p[51]/code[2]"}, {"position_id": 2289, "txt": " again to see the current cache:", "x": "/html/body/div/div/div/article/p[51]/code[2]"}]}, "tags": [], "label": "content", "web_segment_id": 90, "global_sentence_id": 340, "edu_l1_label": "IOS"}, {"txt": "$ ldconfig -r /home/fangpen/.local/share/containers/storage/overlay/3e2b7a6d05ca228fafe39b47a19952d4b81a9c413618c66a7013d794e4db3d96/merged -C /etc/ld.so.cache -f /etc/ld.so.conf -p64 libs found in cache `/etc/ld.so.cache'        ( ... omit ...)        libc.so.6 (libc6,x86-64) => /nix/store/nqb2ns2d1lahnd5ncwmn6k84qfd7vx2k-glibc-2.40-36/lib/libc.so.6        ( ... omit ...)", "language": "english", "position": {"atoms": [{"position_id": 2291, "txt": "$ ldconfig -r /home/fangpen/.local/share/containers/storage/overlay/3e2b7a6d05ca228fafe39b47a19952d4b81a9c413618c66a7013d794e4db3d96/merged -C /etc/ld.so.cache -f /etc/ld.so.conf -p\n64 libs found in cache `/etc/ld.so.cache'\n        ( ... omit ...)\n        libc.so.6 (libc6,x86-64) => /nix/store/nqb2ns2d1lahnd5ncwmn6k84qfd7vx2k-glibc-2.40-36/lib/libc.so.6\n        ( ... omit ...)", "x": "/html/body/div/div/div/article/div[15]/div/pre/code"}]}, "tags": [], "label": "code", "web_segment_id": 91, "global_sentence_id": 341, "edu_l1_label": "EDU_O"}, {"txt": "Cache generated by:", "language": "english", "position": {"atoms": [{"position_id": 2292, "txt": "\nCache generated by:", "x": "/html/body/div/div/div/article/div[15]/div/pre/code"}]}, "tags": [], "label": "code", "web_segment_id": 91, "global_sentence_id": 342, "edu_l1_label": "EDU_O"}, {"txt": " ldconfig (GNU libc) stable release version 2.40", "language": "english", "position": {"atoms": [{"position_id": 2293, "txt": " ldconfig (GNU libc) stable release version 2.40\n", "x": "/html/body/div/div/div/article/div[15]/div/pre/code"}]}, "tags": [], "label": "code", "web_segment_id": 91, "global_sentence_id": 343, "edu_l1_label": "EDU_O"}, {"txt": "Oops.", "language": "english", "position": {"atoms": [{"position_id": 2295, "txt": "Oops.", "x": "/html/body/div/div/div/article/p[52]"}]}, "tags": [], "label": "content", "web_segment_id": 92, "global_sentence_id": 344, "edu_l1_label": "IOS"}, {"txt": " All the caches are pointing to libraries injected by the Nvidia runtime container, including glibc.", "language": "english", "position": {"atoms": [{"position_id": 2296, "txt": " All the caches are pointing to libraries injected by the Nvidia runtime container, including glibc.", "x": "/html/body/div/div/div/article/p[52]"}]}, "tags": [], "label": "content", "web_segment_id": 92, "global_sentence_id": 345, "edu_l1_label": "IOS"}, {"txt": " I guess the glibc compiled inside the plugin container image is different from the one shipped with the Nvidia driver in the Nix store.", "language": "english", "position": {"atoms": [{"position_id": 2297, "txt": " I guess the glibc compiled inside the plugin container image is different from the one shipped with the Nvidia driver in the Nix store.", "x": "/html/body/div/div/div/article/p[52]"}]}, "tags": [], "label": "content", "web_segment_id": 92, "global_sentence_id": 346, "edu_l1_label": "IOS"}, {"txt": " As a result, any executable that relies on glibc would then link to the wrong one, and literally, almost all executables would rely on glibc, and therefore, they all crash immediately.", "language": "english", "position": {"atoms": [{"position_id": 2298, "txt": " As a result, any executable that relies on glibc would then link to the wrong one, and literally, almost all executables would rely on glibc, and therefore, they all crash immediately.", "x": "/html/body/div/div/div/article/p[52]"}]}, "tags": [], "label": "content", "web_segment_id": 92, "global_sentence_id": 347, "edu_l1_label": "IOS"}, {"txt": "After learning the root cause, I quickly drafted an issue, reported it to Nvidia’s nvidia-container-toolkit repository, and found an easy fix.", "language": "english", "position": {"atoms": [{"position_id": 2300, "txt": "After learning the root cause, I quickly drafted ", "x": "/html/body/div/div/div/article/p[53]"}, {"position_id": 2302, "txt": "an issue", "x": "/html/body/div/div/div/article/p[53]/a[1]"}, {"position_id": 2304, "txt": ", reported it to Nvidia’s nvidia-container-toolkit repository, and found an easy fix.", "x": "/html/body/div/div/div/article/p[53]/a[1]"}]}, "tags": [], "label": "content", "web_segment_id": 93, "global_sentence_id": 348, "edu_l1_label": "IOS"}, {"txt": " Most containers come with the config file in /etc/ld.so.conf.d pointing to the existing library paths.", "language": "english", "position": {"atoms": [{"position_id": 2305, "txt": " Most containers come with the config file in ", "x": "/html/body/div/div/div/article/p[53]/a[1]"}, {"position_id": 2307, "txt": "/etc/ld.so.conf.d", "x": "/html/body/div/div/div/article/p[53]/code[1]"}, {"position_id": 2309, "txt": " pointing to the existing library paths.", "x": "/html/body/div/div/div/article/p[53]/code[1]"}]}, "tags": [], "label": "content", "web_segment_id": 93, "global_sentence_id": 349, "edu_l1_label": "IOS"}, {"txt": " The Nvidia runtime hook generates its own config file pointing to the library path provided in the argument.", "language": "english", "position": {"atoms": [{"position_id": 2310, "txt": " The Nvidia runtime hook generates its own config file pointing to the library path provided in the argument.", "x": "/html/body/div/div/div/article/p[53]/code[1]"}]}, "tags": [], "label": "content", "web_segment_id": 93, "global_sentence_id": 350, "edu_l1_label": "IOS"}, {"txt": " But without the one pointing to the existing library path, there will only be a cache pointing to the injected lib.", "language": "english", "position": {"atoms": [{"position_id": 2311, "txt": " But without the one pointing to the existing library path, there will only be a cache pointing to the injected lib.", "x": "/html/body/div/div/div/article/p[53]/code[1]"}]}, "tags": [], "label": "content", "web_segment_id": 93, "global_sentence_id": 351, "edu_l1_label": "IOS"}, {"txt": " To solve the problem, I created a PR with a config file pointing to /lib64.", "language": "english", "position": {"atoms": [{"position_id": 2312, "txt": " To solve the problem, I ", "x": "/html/body/div/div/div/article/p[53]/code[1]"}, {"position_id": 2314, "txt": "created a PR", "x": "/html/body/div/div/div/article/p[53]/a[2]"}, {"position_id": 2316, "txt": " with a config file pointing to ", "x": "/html/body/div/div/div/article/p[53]/a[2]"}, {"position_id": 2318, "txt": "/lib64", "x": "/html/body/div/div/div/article/p[53]/code[2]"}, {"position_id": 2320, "txt": ".", "x": "/html/body/div/div/div/article/p[53]/code[2]"}]}, "tags": [], "label": "content", "web_segment_id": 93, "global_sentence_id": 352, "edu_l1_label": "IOS"}, {"txt": "RUN echo\"/lib64\"> /etc/ld.so.conf.d/lib64.conf", "language": "english", "position": {"atoms": [{"position_id": 2322, "txt": "RUN ", "x": "/html/body/div/div/div/article/div[16]/div/pre/code/span[1]"}, {"position_id": 2324, "txt": "echo", "x": "/html/body/div/div/div/article/div[16]/div/pre/code/span[2]"}, {"position_id": 2326, "txt": "\"/lib64\"", "x": "/html/body/div/div/div/article/div[16]/div/pre/code/span[3]"}, {"position_id": 2328, "txt": ">", "x": "/html/body/div/div/div/article/div[16]/div/pre/code/span[4]"}, {"position_id": 2330, "txt": " /etc/ld.so.conf.d/lib64.conf\n", "x": "/html/body/div/div/div/article/div[16]/div/pre/code/span[4]"}]}, "tags": [], "label": "code", "web_segment_id": 94, "global_sentence_id": 353, "edu_l1_label": "EDU_O"}, {"txt": "It was just a line code change, and it took me a few days to find out, including building the nix-playground tool 😅 That’s why one should never measure productivity with the number of lines added.", "language": "english", "position": {"atoms": [{"position_id": 2332, "txt": "It was just a line code change, and it took me a few days to find out, including building the nix-playground tool 😅 That’s why one should never measure productivity with the number of lines added.", "x": "/html/body/div/div/div/article/p[54]"}]}, "tags": [], "label": "content", "web_segment_id": 95, "global_sentence_id": 354, "edu_l1_label": "IOS"}, {"txt": "The missing RuntimeClass", "language": "english", "position": {"atoms": [{"position_id": 2334, "txt": "The missing RuntimeClass", "x": "/html/body/div/div/div/article/h2[6]"}]}, "tags": ["h2"], "label": "title1", "web_segment_id": 96, "global_sentence_id": 355, "edu_l1_label": "BOS"}, {"txt": "Suppose you follow many online guides about running Kubernetes pods with Nvidia drivers.", "language": "english", "position": {"atoms": [{"position_id": 2336, "txt": "Suppose you follow many online guides about running Kubernetes pods with Nvidia drivers.", "x": "/html/body/div/div/div/article/p[55]"}]}, "tags": [], "label": "content", "web_segment_id": 97, "global_sentence_id": 356, "edu_l1_label": "IOS"}, {"txt": " In that case, you may think you should be able to run it after you update the Containerd configuration to include the Nvidia CRI and set it as the default like this:", "language": "english", "position": {"atoms": [{"position_id": 2337, "txt": " In that case, you may think you should be able to run it after you update the Containerd configuration to include the Nvidia CRI and set it as the default like this:", "x": "/html/body/div/div/div/article/p[55]"}]}, "tags": [], "label": "content", "web_segment_id": 97, "global_sentence_id": 357, "edu_l1_label": "IOS"}, {"txt": "[plugins.\"io.containerd.grpc.v1.cri\"]default_runtime_name=\"nvidia\"# other stuff ..", "language": "english", "position": {"atoms": [{"position_id": 2339, "txt": "[plugins.\"io.containerd.grpc.v1.cri\"]", "x": "/html/body/div/div/div/article/div[17]/div/pre/code/span[1]"}, {"position_id": 2341, "txt": "default_runtime_name", "x": "/html/body/div/div/div/article/div[17]/div/pre/code/span[2]"}, {"position_id": 2343, "txt": "=", "x": "/html/body/div/div/div/article/div[17]/div/pre/code/span[3]"}, {"position_id": 2345, "txt": "\"nvidia\"", "x": "/html/body/div/div/div/article/div[17]/div/pre/code/span[4]"}, {"position_id": 2347, "txt": "# other stuff ..", "x": "/html/body/div/div/div/article/div[17]/div/pre/code/span[5]"}]}, "tags": [], "label": "code", "web_segment_id": 98, "global_sentence_id": 358, "edu_l1_label": "EDU_O"}, {"txt": "Unfortunately, it doesn’t work.", "language": "english", "position": {"atoms": [{"position_id": 2349, "txt": "Unfortunately, it doesn’t work.", "x": "/html/body/div/div/div/article/p[56]"}]}, "tags": [], "label": "content", "web_segment_id": 99, "global_sentence_id": 359, "edu_l1_label": "IOS"}, {"txt": " I haven’t dug into this yet, but according to Nvidia’s k8s-device-plugin readme page:", "language": "english", "position": {"atoms": [{"position_id": 2350, "txt": " I haven’t dug into this yet, but according to Nvidia’s k8s-device-plugin readme page:", "x": "/html/body/div/div/div/article/p[56]"}]}, "tags": [], "label": "content", "web_segment_id": 99, "global_sentence_id": 360, "edu_l1_label": "IOS"}, {"txt": "If the nvidia runtime should be set as the default runtime (with non-cri docker versions, for example), the –set-as-default argument must also be included in the commands above.", "language": "english", "position": {"atoms": [{"position_id": 2352, "txt": "If the nvidia runtime should be set as the default runtime (with non-cri docker versions, for example), the –set-as-default argument must also be included in the commands above.", "x": "/html/body/div/div/div/article/blockquote[2]/p"}]}, "tags": [], "label": "content", "web_segment_id": 100, "global_sentence_id": 361, "edu_l1_label": "IOS"}, {"txt": " If this is not done, a RuntimeClass needs to be defined.", "language": "english", "position": {"atoms": [{"position_id": 2353, "txt": " If this is not done, a RuntimeClass needs to be defined.", "x": "/html/body/div/div/div/article/blockquote[2]/p"}]}, "tags": [], "label": "content", "web_segment_id": 100, "global_sentence_id": 362, "edu_l1_label": "IOS"}, {"txt": "It’s just one line in the README, and they’re very easy to overlook.", "language": "english", "position": {"atoms": [{"position_id": 2355, "txt": "It’s just one line in the README, and they’re very easy to overlook.", "x": "/html/body/div/div/div/article/p[57]"}]}, "tags": [], "label": "content", "web_segment_id": 101, "global_sentence_id": 363, "edu_l1_label": "IOS"}, {"txt": " So, without an explicit RuntimeClass defined and assigned to a pod in Kubernetes, it won’t pick up the Nvidia runtime.", "language": "english", "position": {"atoms": [{"position_id": 2356, "txt": " So, without an explicit ", "x": "/html/body/div/div/div/article/p[57]"}, {"position_id": 2358, "txt": "RuntimeClass", "x": "/html/body/div/div/div/article/p[57]/a"}, {"position_id": 2360, "txt": " defined and assigned to a pod in Kubernetes, it won’t pick up the Nvidia runtime.", "x": "/html/body/div/div/div/article/p[57]/a"}]}, "tags": [], "label": "content", "web_segment_id": 101, "global_sentence_id": 364, "edu_l1_label": "IOS"}, {"txt": " I didn’t have time to find out how to make a runtime default on a node yet, but for now, I had to define a RuntimeClass like this:", "language": "english", "position": {"atoms": [{"position_id": 2361, "txt": " I didn’t have time to find out how to make a runtime default on a node yet, but for now, I had to define a RuntimeClass like this:", "x": "/html/body/div/div/div/article/p[57]/a"}]}, "tags": [], "label": "content", "web_segment_id": 101, "global_sentence_id": 365, "edu_l1_label": "IOS"}, {"txt": "apiVersion:", "language": "english", "position": {"atoms": [{"position_id": 2363, "txt": "apiVersion", "x": "/html/body/div/div/div/article/div[18]/div/pre/code/span[1]"}, {"position_id": 2365, "txt": ":", "x": "/html/body/div/div/div/article/div[18]/div/pre/code/span[2]"}]}, "tags": [], "label": "code", "web_segment_id": 102, "global_sentence_id": 366, "edu_l1_label": "EDU_O"}, {"txt": "node.k8s.io/v1kind:", "language": "english", "position": {"atoms": [{"position_id": 2367, "txt": "node.k8s.io/v1", "x": "/html/body/div/div/div/article/div[18]/div/pre/code/span[3]"}, {"position_id": 2369, "txt": "kind", "x": "/html/body/div/div/div/article/div[18]/div/pre/code/span[4]"}, {"position_id": 2371, "txt": ":", "x": "/html/body/div/div/div/article/div[18]/div/pre/code/span[5]"}]}, "tags": [], "label": "code", "web_segment_id": 102, "global_sentence_id": 367, "edu_l1_label": "EDU_O"}, {"txt": "RuntimeClasshandler:", "language": "english", "position": {"atoms": [{"position_id": 2373, "txt": "RuntimeClass", "x": "/html/body/div/div/div/article/div[18]/div/pre/code/span[6]"}, {"position_id": 2375, "txt": "handler", "x": "/html/body/div/div/div/article/div[18]/div/pre/code/span[7]"}, {"position_id": 2377, "txt": ":", "x": "/html/body/div/div/div/article/div[18]/div/pre/code/span[8]"}]}, "tags": [], "label": "code", "web_segment_id": 102, "global_sentence_id": 368, "edu_l1_label": "EDU_O"}, {"txt": "nvidiametadata:", "language": "english", "position": {"atoms": [{"position_id": 2379, "txt": "nvidia", "x": "/html/body/div/div/div/article/div[18]/div/pre/code/span[9]"}, {"position_id": 2381, "txt": "metadata", "x": "/html/body/div/div/div/article/div[18]/div/pre/code/span[10]"}, {"position_id": 2383, "txt": ":", "x": "/html/body/div/div/div/article/div[18]/div/pre/code/span[11]"}]}, "tags": [], "label": "code", "web_segment_id": 102, "global_sentence_id": 369, "edu_l1_label": "EDU_O"}, {"txt": "name:", "language": "english", "position": {"atoms": [{"position_id": 2385, "txt": "name", "x": "/html/body/div/div/div/article/div[18]/div/pre/code/span[12]"}, {"position_id": 2387, "txt": ":", "x": "/html/body/div/div/div/article/div[18]/div/pre/code/span[13]"}]}, "tags": [], "label": "code", "web_segment_id": 102, "global_sentence_id": 370, "edu_l1_label": "EDU_O"}, {"txt": "nvidia", "language": "english", "position": {"atoms": [{"position_id": 2389, "txt": "nvidia", "x": "/html/body/div/div/div/article/div[18]/div/pre/code/span[14]"}]}, "tags": [], "label": "code", "web_segment_id": 102, "global_sentence_id": 371, "edu_l1_label": "EDU_O"}, {"txt": "Then, I will assign the runtimeClassName to it so that it will pick up my CRI configuration for Containerd.", "language": "english", "position": {"atoms": [{"position_id": 2391, "txt": "Then, I will assign the ", "x": "/html/body/div/div/div/article/p[58]"}, {"position_id": 2393, "txt": "runtimeClassName", "x": "/html/body/div/div/div/article/p[58]/code"}, {"position_id": 2395, "txt": " to it so that it will pick up my CRI configuration for Containerd.", "x": "/html/body/div/div/div/article/p[58]/code"}]}, "tags": [], "label": "content", "web_segment_id": 103, "global_sentence_id": 372, "edu_l1_label": "IOS"}, {"txt": "apiVersion:", "language": "english", "position": {"atoms": [{"position_id": 2397, "txt": "apiVersion", "x": "/html/body/div/div/div/article/div[19]/div/pre/code/span[1]"}, {"position_id": 2399, "txt": ":", "x": "/html/body/div/div/div/article/div[19]/div/pre/code/span[2]"}]}, "tags": [], "label": "code", "web_segment_id": 104, "global_sentence_id": 373, "edu_l1_label": "EDU_O"}, {"txt": "v1kind:", "language": "english", "position": {"atoms": [{"position_id": 2401, "txt": "v1", "x": "/html/body/div/div/div/article/div[19]/div/pre/code/span[3]"}, {"position_id": 2403, "txt": "kind", "x": "/html/body/div/div/div/article/div[19]/div/pre/code/span[4]"}, {"position_id": 2405, "txt": ":", "x": "/html/body/div/div/div/article/div[19]/div/pre/code/span[5]"}]}, "tags": [], "label": "code", "web_segment_id": 104, "global_sentence_id": 374, "edu_l1_label": "EDU_O"}, {"txt": "Podmetadata:", "language": "english", "position": {"atoms": [{"position_id": 2407, "txt": "Pod", "x": "/html/body/div/div/div/article/div[19]/div/pre/code/span[6]"}, {"position_id": 2409, "txt": "metadata", "x": "/html/body/div/div/div/article/div[19]/div/pre/code/span[7]"}, {"position_id": 2411, "txt": ":", "x": "/html/body/div/div/div/article/div[19]/div/pre/code/span[8]"}]}, "tags": [], "label": "code", "web_segment_id": 104, "global_sentence_id": 375, "edu_l1_label": "EDU_O"}, {"txt": "name:", "language": "english", "position": {"atoms": [{"position_id": 2413, "txt": "name", "x": "/html/body/div/div/div/article/div[19]/div/pre/code/span[9]"}, {"position_id": 2415, "txt": ":", "x": "/html/body/div/div/div/article/div[19]/div/pre/code/span[10]"}]}, "tags": [], "label": "code", "web_segment_id": 104, "global_sentence_id": 376, "edu_l1_label": "EDU_O"}, {"txt": "gpu-podspec:", "language": "english", "position": {"atoms": [{"position_id": 2417, "txt": "gpu-pod", "x": "/html/body/div/div/div/article/div[19]/div/pre/code/span[11]"}, {"position_id": 2419, "txt": "spec", "x": "/html/body/div/div/div/article/div[19]/div/pre/code/span[12]"}, {"position_id": 2421, "txt": ":", "x": "/html/body/div/div/div/article/div[19]/div/pre/code/span[13]"}]}, "tags": [], "label": "code", "web_segment_id": 104, "global_sentence_id": 377, "edu_l1_label": "EDU_O"}, {"txt": "# the `runtimeClassName` is needed, otherwise default runtime will still be usedruntimeClassName:", "language": "english", "position": {"atoms": [{"position_id": 2423, "txt": "# the `runtimeClassName` is needed, otherwise default runtime will still be used", "x": "/html/body/div/div/div/article/div[19]/div/pre/code/span[14]"}, {"position_id": 2425, "txt": "runtimeClassName", "x": "/html/body/div/div/div/article/div[19]/div/pre/code/span[15]"}, {"position_id": 2427, "txt": ":", "x": "/html/body/div/div/div/article/div[19]/div/pre/code/span[16]"}]}, "tags": [], "label": "code", "web_segment_id": 104, "global_sentence_id": 378, "edu_l1_label": "EDU_O"}, {"txt": "nvidiacontainers:", "language": "english", "position": {"atoms": [{"position_id": 2429, "txt": "nvidia", "x": "/html/body/div/div/div/article/div[19]/div/pre/code/span[17]"}, {"position_id": 2431, "txt": "containers", "x": "/html/body/div/div/div/article/div[19]/div/pre/code/span[18]"}, {"position_id": 2433, "txt": ":", "x": "/html/body/div/div/div/article/div[19]/div/pre/code/span[19]"}]}, "tags": [], "label": "code", "web_segment_id": 104, "global_sentence_id": 379, "edu_l1_label": "EDU_O"}, {"txt": "-name:", "language": "english", "position": {"atoms": [{"position_id": 2435, "txt": "-", "x": "/html/body/div/div/div/article/div[19]/div/pre/code/span[20]"}, {"position_id": 2437, "txt": "name", "x": "/html/body/div/div/div/article/div[19]/div/pre/code/span[21]"}, {"position_id": 2439, "txt": ":", "x": "/html/body/div/div/div/article/div[19]/div/pre/code/span[22]"}]}, "tags": [], "label": "code", "web_segment_id": 104, "global_sentence_id": 380, "edu_l1_label": "EDU_O"}, {"txt": "cuda-containerimage:", "language": "english", "position": {"atoms": [{"position_id": 2441, "txt": "cuda-container", "x": "/html/body/div/div/div/article/div[19]/div/pre/code/span[23]"}, {"position_id": 2443, "txt": "image", "x": "/html/body/div/div/div/article/div[19]/div/pre/code/span[24]"}, {"position_id": 2445, "txt": ":", "x": "/html/body/div/div/div/article/div[19]/div/pre/code/span[25]"}]}, "tags": [], "label": "code", "web_segment_id": 104, "global_sentence_id": 381, "edu_l1_label": "EDU_O"}, {"txt": "nvcr.io/nvidia/k8s/cuda-sample:", "language": "english", "position": {"atoms": [{"position_id": 2447, "txt": "nvcr.io/nvidia/k8s/cuda-sample:", "x": "/html/body/div/div/div/article/div[19]/div/pre/code/span[26]"}]}, "tags": [], "label": "code", "web_segment_id": 104, "global_sentence_id": 382, "edu_l1_label": "EDU_O"}, {"txt": "vectoradd-cuda12.5.0resources:", "language": "english", "position": {"atoms": [{"position_id": 2448, "txt": "vectoradd-cuda12.5.0", "x": "/html/body/div/div/div/article/div[19]/div/pre/code/span[26]"}, {"position_id": 2450, "txt": "resources", "x": "/html/body/div/div/div/article/div[19]/div/pre/code/span[27]"}, {"position_id": 2452, "txt": ":", "x": "/html/body/div/div/div/article/div[19]/div/pre/code/span[28]"}]}, "tags": [], "label": "code", "web_segment_id": 104, "global_sentence_id": 383, "edu_l1_label": "EDU_O"}, {"txt": "limits:", "language": "english", "position": {"atoms": [{"position_id": 2454, "txt": "limits", "x": "/html/body/div/div/div/article/div[19]/div/pre/code/span[29]"}, {"position_id": 2456, "txt": ":", "x": "/html/body/div/div/div/article/div[19]/div/pre/code/span[30]"}]}, "tags": [], "label": "code", "web_segment_id": 104, "global_sentence_id": 384, "edu_l1_label": "EDU_O"}, {"txt": "nvidia.com/gpu:", "language": "english", "position": {"atoms": [{"position_id": 2458, "txt": "nvidia.com/gpu", "x": "/html/body/div/div/div/article/div[19]/div/pre/code/span[31]"}, {"position_id": 2460, "txt": ":", "x": "/html/body/div/div/div/article/div[19]/div/pre/code/span[32]"}]}, "tags": [], "label": "code", "web_segment_id": 104, "global_sentence_id": 385, "edu_l1_label": "EDU_O"}, {"txt": "1# requesting 1 GPUtolerations:", "language": "english", "position": {"atoms": [{"position_id": 2462, "txt": "1", "x": "/html/body/div/div/div/article/div[19]/div/pre/code/span[33]"}, {"position_id": 2464, "txt": "# requesting 1 GPU", "x": "/html/body/div/div/div/article/div[19]/div/pre/code/span[34]"}, {"position_id": 2466, "txt": "tolerations", "x": "/html/body/div/div/div/article/div[19]/div/pre/code/span[35]"}, {"position_id": 2468, "txt": ":", "x": "/html/body/div/div/div/article/div[19]/div/pre/code/span[36]"}]}, "tags": [], "label": "code", "web_segment_id": 104, "global_sentence_id": 386, "edu_l1_label": "EDU_O"}, {"txt": "-key:", "language": "english", "position": {"atoms": [{"position_id": 2470, "txt": "-", "x": "/html/body/div/div/div/article/div[19]/div/pre/code/span[37]"}, {"position_id": 2472, "txt": "key", "x": "/html/body/div/div/div/article/div[19]/div/pre/code/span[38]"}, {"position_id": 2474, "txt": ":", "x": "/html/body/div/div/div/article/div[19]/div/pre/code/span[39]"}]}, "tags": [], "label": "code", "web_segment_id": 104, "global_sentence_id": 387, "edu_l1_label": "EDU_O"}, {"txt": "nvidia.com/gpuoperator:", "language": "english", "position": {"atoms": [{"position_id": 2476, "txt": "nvidia.com/gpu", "x": "/html/body/div/div/div/article/div[19]/div/pre/code/span[40]"}, {"position_id": 2478, "txt": "operator", "x": "/html/body/div/div/div/article/div[19]/div/pre/code/span[41]"}, {"position_id": 2480, "txt": ":", "x": "/html/body/div/div/div/article/div[19]/div/pre/code/span[42]"}]}, "tags": [], "label": "code", "web_segment_id": 104, "global_sentence_id": 388, "edu_l1_label": "EDU_O"}, {"txt": "Existseffect:", "language": "english", "position": {"atoms": [{"position_id": 2482, "txt": "Exists", "x": "/html/body/div/div/div/article/div[19]/div/pre/code/span[43]"}, {"position_id": 2484, "txt": "effect", "x": "/html/body/div/div/div/article/div[19]/div/pre/code/span[44]"}, {"position_id": 2486, "txt": ":", "x": "/html/body/div/div/div/article/div[19]/div/pre/code/span[45]"}]}, "tags": [], "label": "code", "web_segment_id": 104, "global_sentence_id": 389, "edu_l1_label": "EDU_O"}, {"txt": "NoSchedule", "language": "english", "position": {"atoms": [{"position_id": 2488, "txt": "NoSchedule", "x": "/html/body/div/div/div/article/div[19]/div/pre/code/span[46]"}]}, "tags": [], "label": "code", "web_segment_id": 104, "global_sentence_id": 390, "edu_l1_label": "EDU_O"}, {"txt": "Yet another rabbit hole - PyCharm cannot open my WSL NixOS anymore", "language": "english", "position": {"atoms": [{"position_id": 2490, "txt": "Yet another rabbit hole - PyCharm cannot open my WSL NixOS anymore", "x": "/html/body/div/div/div/article/h2[7]"}]}, "tags": ["h2"], "label": "title1", "web_segment_id": 105, "global_sentence_id": 391, "edu_l1_label": "BOS"}, {"txt": "Interestingly, after the recent update, my PyCharm stopped working with my Python projects in my NixOS distro, WSL (Windows Subsystem Linux).", "language": "english", "position": {"atoms": [{"position_id": 2492, "txt": "Interestingly, after the recent update, my ", "x": "/html/body/div/div/div/article/p[59]"}, {"position_id": 2494, "txt": "PyCharm", "x": "/html/body/div/div/div/article/p[59]/a[1]"}, {"position_id": 2496, "txt": " stopped working with my Python projects in my NixOS distro, ", "x": "/html/body/div/div/div/article/p[59]/a[1]"}, {"position_id": 2498, "txt": "WSL (Windows Subsystem Linux)", "x": "/html/body/div/div/div/article/p[59]/a[2]"}, {"position_id": 2500, "txt": ".", "x": "/html/body/div/div/div/article/p[59]/a[2]"}]}, "tags": [], "label": "content", "web_segment_id": 106, "global_sentence_id": 392, "edu_l1_label": "IOS"}, {"txt": " While it sounds a bit unrelated to the main focus of getting the Nvidia GPU to run on Kubernetes, I needed to build nix-playground in Python, and it can only work in a Linux environment.", "language": "english", "position": {"atoms": [{"position_id": 2501, "txt": " While it sounds a bit unrelated to the main focus of getting the Nvidia GPU to run on Kubernetes, I needed to build nix-playground in Python, and it can only work in a Linux environment.", "x": "/html/body/div/div/div/article/p[59]/a[2]"}]}, "tags": [], "label": "content", "web_segment_id": 106, "global_sentence_id": 393, "edu_l1_label": "IOS"}, {"txt": " I am bringing this up because this is the third level deep rabbit hole from Nvidia GPU on Kubernetes.", "language": "english", "position": {"atoms": [{"position_id": 2502, "txt": " I am bringing this up because this is the third level deep rabbit hole from Nvidia GPU on Kubernetes.", "x": "/html/body/div/div/div/article/p[59]/a[2]"}]}, "tags": [], "label": "content", "web_segment_id": 106, "global_sentence_id": 394, "edu_l1_label": "IOS"}, {"txt": "I reported this problem a while back, but I decided to dig this hole while building nix-playground because not being able to use PyCharm with WSL projects greatly impacts my productivity.", "language": "english", "position": {"atoms": [{"position_id": 2504, "txt": "I ", "x": "/html/body/div/div/div/article/p[60]"}, {"position_id": 2506, "txt": "reported this problem", "x": "/html/body/div/div/div/article/p[60]/a"}, {"position_id": 2508, "txt": " a while back, but I decided to dig this hole while building nix-playground because not being able to use PyCharm with WSL projects greatly impacts my productivity.", "x": "/html/body/div/div/div/article/p[60]/a"}]}, "tags": [], "label": "content", "web_segment_id": 107, "global_sentence_id": 395, "edu_l1_label": "IOS"}, {"txt": " After digging briefly, I realized that PyCharm expected many executables at certain locations, such as in /usr/bin.", "language": "english", "position": {"atoms": [{"position_id": 2509, "txt": " After digging briefly, I realized that PyCharm expected many executables at certain locations, such as in ", "x": "/html/body/div/div/div/article/p[60]/a"}, {"position_id": 2511, "txt": "/usr/bin", "x": "/html/body/div/div/div/article/p[60]/code[1]"}, {"position_id": 2513, "txt": ".", "x": "/html/body/div/div/div/article/p[60]/code[1]"}]}, "tags": [], "label": "content", "web_segment_id": 107, "global_sentence_id": 396, "edu_l1_label": "IOS"}, {"txt": " I addressed the problem by adding them to WSL NixOS config’s extraBin.", "language": "english", "position": {"atoms": [{"position_id": 2514, "txt": " I addressed the problem by adding them to WSL NixOS config’s ", "x": "/html/body/div/div/div/article/p[60]/code[1]"}, {"position_id": 2516, "txt": "extraBin", "x": "/html/body/div/div/div/article/p[60]/code[2]"}, {"position_id": 2518, "txt": ".", "x": "/html/body/div/div/div/article/p[60]/code[2]"}]}, "tags": [], "label": "content", "web_segment_id": 107, "global_sentence_id": 397, "edu_l1_label": "IOS"}, {"txt": "{config,lib,pkgs,...}:", "language": "english", "position": {"atoms": [{"position_id": 2520, "txt": "{", "x": "/html/body/div/div/div/article/div[20]/div/pre/code/span[1]"}, {"position_id": 2522, "txt": "config", "x": "/html/body/div/div/div/article/div[20]/div/pre/code/span[2]"}, {"position_id": 2524, "txt": ",", "x": "/html/body/div/div/div/article/div[20]/div/pre/code/span[3]"}, {"position_id": 2526, "txt": "lib", "x": "/html/body/div/div/div/article/div[20]/div/pre/code/span[4]"}, {"position_id": 2528, "txt": ",", "x": "/html/body/div/div/div/article/div[20]/div/pre/code/span[5]"}, {"position_id": 2530, "txt": "pkgs", "x": "/html/body/div/div/div/article/div[20]/div/pre/code/span[6]"}, {"position_id": 2532, "txt": ",", "x": "/html/body/div/div/div/article/div[20]/div/pre/code/span[7]"}, {"position_id": 2534, "txt": "...", "x": "/html/body/div/div/div/article/div[20]/div/pre/code/span[8]"}, {"position_id": 2536, "txt": "}:", "x": "/html/body/div/div/div/article/div[20]/div/pre/code/span[9]"}]}, "tags": [], "label": "code", "web_segment_id": 108, "global_sentence_id": 398, "edu_l1_label": "EDU_O"}, {"txt": "{wsl={enable=true;extraBin=withpkgs;[{src=\"${coreutils}/bin/mkdir\";}{src=\"${coreutils}/bin/cat\";}{src=\"${coreutils}/bin/whoami\";}{src=\"${coreutils}/bin/ls\";}{src=\"${busybox}/bin/addgroup\";}{src=\"${su}/bin/groupadd\";}{src=\"${su}/bin/usermod\";}{src=\"${busybox}/bin/busybox\";}{src=\"${busybox}/bin/chmod\";}{src=\"${busybox}/bin/cp\";}{src=\"${busybox}/bin/cut\";}{src=\"${busybox}/bin/getent\";}{src=\"${busybox}/bin/head\";}{src=\"${busybox}/bin/mktemp\";}{src=\"${busybox}/bin/rm\";}{src=\"${busybox}/bin/sed\";}{src=\"${busybox}/bin/tail\";}{src=\"${busybox}/bin/uname\";}{src=\"${busybox}/bin/which\";}];};", "language": "english", "position": {"atoms": [{"position_id": 2538, "txt": "{", "x": "/html/body/div/div/div/article/div[20]/div/pre/code/span[10]"}, {"position_id": 2540, "txt": "wsl", "x": "/html/body/div/div/div/article/div[20]/div/pre/code/span[11]"}, {"position_id": 2542, "txt": "=", "x": "/html/body/div/div/div/article/div[20]/div/pre/code/span[12]"}, {"position_id": 2544, "txt": "{", "x": "/html/body/div/div/div/article/div[20]/div/pre/code/span[13]"}, {"position_id": 2546, "txt": "enable", "x": "/html/body/div/div/div/article/div[20]/div/pre/code/span[14]"}, {"position_id": 2548, "txt": "=", "x": "/html/body/div/div/div/article/div[20]/div/pre/code/span[15]"}, {"position_id": 2550, "txt": "true", "x": "/html/body/div/div/div/article/div[20]/div/pre/code/span[16]"}, {"position_id": 2552, "txt": ";", "x": "/html/body/div/div/div/article/div[20]/div/pre/code/span[17]"}, {"position_id": 2554, "txt": "extraBin", "x": "/html/body/div/div/div/article/div[20]/div/pre/code/span[18]"}, {"position_id": 2556, "txt": "=", "x": "/html/body/div/div/div/article/div[20]/div/pre/code/span[19]"}, {"position_id": 2558, "txt": "with", "x": "/html/body/div/div/div/article/div[20]/div/pre/code/span[20]"}, {"position_id": 2560, "txt": "pkgs", "x": "/html/body/div/div/div/article/div[20]/div/pre/code/span[21]"}, {"position_id": 2562, "txt": ";", "x": "/html/body/div/div/div/article/div[20]/div/pre/code/span[22]"}, {"position_id": 2564, "txt": "[", "x": "/html/body/div/div/div/article/div[20]/div/pre/code/span[23]"}, {"position_id": 2566, "txt": "{", "x": "/html/body/div/div/div/article/div[20]/div/pre/code/span[24]"}, {"position_id": 2568, "txt": "src", "x": "/html/body/div/div/div/article/div[20]/div/pre/code/span[25]"}, {"position_id": 2570, "txt": "=", "x": "/html/body/div/div/div/article/div[20]/div/pre/code/span[26]"}, {"position_id": 2572, "txt": "\"", "x": "/html/body/div/div/div/article/div[20]/div/pre/code/span[27]"}, {"position_id": 2574, "txt": "${", "x": "/html/body/div/div/div/article/div[20]/div/pre/code/span[28]"}, {"position_id": 2576, "txt": "coreutils", "x": "/html/body/div/div/div/article/div[20]/div/pre/code/span[29]"}, {"position_id": 2578, "txt": "}", "x": "/html/body/div/div/div/article/div[20]/div/pre/code/span[30]"}, {"position_id": 2580, "txt": "/bin/mkdir\"", "x": "/html/body/div/div/div/article/div[20]/div/pre/code/span[31]"}, {"position_id": 2582, "txt": ";}", "x": "/html/body/div/div/div/article/div[20]/div/pre/code/span[32]"}, {"position_id": 2584, "txt": "{", "x": "/html/body/div/div/div/article/div[20]/div/pre/code/span[33]"}, {"position_id": 2586, "txt": "src", "x": "/html/body/div/div/div/article/div[20]/div/pre/code/span[34]"}, {"position_id": 2588, "txt": "=", "x": "/html/body/div/div/div/article/div[20]/div/pre/code/span[35]"}, {"position_id": 2590, "txt": "\"", "x": "/html/body/div/div/div/article/div[20]/div/pre/code/span[36]"}, {"position_id": 2592, "txt": "${", "x": "/html/body/div/div/div/article/div[20]/div/pre/code/span[37]"}, {"position_id": 2594, "txt": "coreutils", "x": "/html/body/div/div/div/article/div[20]/div/pre/code/span[38]"}, {"position_id": 2596, "txt": "}", "x": "/html/body/div/div/div/article/div[20]/div/pre/code/span[39]"}, {"position_id": 2598, "txt": "/bin/cat\"", "x": "/html/body/div/div/div/article/div[20]/div/pre/code/span[40]"}, {"position_id": 2600, "txt": ";}", "x": "/html/body/div/div/div/article/div[20]/div/pre/code/span[41]"}, {"position_id": 2602, "txt": "{", "x": "/html/body/div/div/div/article/div[20]/div/pre/code/span[42]"}, {"position_id": 2604, "txt": "src", "x": "/html/body/div/div/div/article/div[20]/div/pre/code/span[43]"}, {"position_id": 2606, "txt": "=", "x": "/html/body/div/div/div/article/div[20]/div/pre/code/span[44]"}, {"position_id": 2608, "txt": "\"", "x": "/html/body/div/div/div/article/div[20]/div/pre/code/span[45]"}, {"position_id": 2610, "txt": "${", "x": "/html/body/div/div/div/article/div[20]/div/pre/code/span[46]"}, {"position_id": 2612, "txt": "coreutils", "x": "/html/body/div/div/div/article/div[20]/div/pre/code/span[47]"}, {"position_id": 2614, "txt": "}", "x": "/html/body/div/div/div/article/div[20]/div/pre/code/span[48]"}, {"position_id": 2616, "txt": "/bin/whoami\"", "x": "/html/body/div/div/div/article/div[20]/div/pre/code/span[49]"}, {"position_id": 2618, "txt": ";}", "x": "/html/body/div/div/div/article/div[20]/div/pre/code/span[50]"}, {"position_id": 2620, "txt": "{", "x": "/html/body/div/div/div/article/div[20]/div/pre/code/span[51]"}, {"position_id": 2622, "txt": "src", "x": "/html/body/div/div/div/article/div[20]/div/pre/code/span[52]"}, {"position_id": 2624, "txt": "=", "x": "/html/body/div/div/div/article/div[20]/div/pre/code/span[53]"}, {"position_id": 2626, "txt": "\"", "x": "/html/body/div/div/div/article/div[20]/div/pre/code/span[54]"}, {"position_id": 2628, "txt": "${", "x": "/html/body/div/div/div/article/div[20]/div/pre/code/span[55]"}, {"position_id": 2630, "txt": "coreutils", "x": "/html/body/div/div/div/article/div[20]/div/pre/code/span[56]"}, {"position_id": 2632, "txt": "}", "x": "/html/body/div/div/div/article/div[20]/div/pre/code/span[57]"}, {"position_id": 2634, "txt": "/bin/ls\"", "x": "/html/body/div/div/div/article/div[20]/div/pre/code/span[58]"}, {"position_id": 2636, "txt": ";}", "x": "/html/body/div/div/div/article/div[20]/div/pre/code/span[59]"}, {"position_id": 2638, "txt": "{", "x": "/html/body/div/div/div/article/div[20]/div/pre/code/span[60]"}, {"position_id": 2640, "txt": "src", "x": "/html/body/div/div/div/article/div[20]/div/pre/code/span[61]"}, {"position_id": 2642, "txt": "=", "x": "/html/body/div/div/div/article/div[20]/div/pre/code/span[62]"}, {"position_id": 2644, "txt": "\"", "x": "/html/body/div/div/div/article/div[20]/div/pre/code/span[63]"}, {"position_id": 2646, "txt": "${", "x": "/html/body/div/div/div/article/div[20]/div/pre/code/span[64]"}, {"position_id": 2648, "txt": "busybox", "x": "/html/body/div/div/div/article/div[20]/div/pre/code/span[65]"}, {"position_id": 2650, "txt": "}", "x": "/html/body/div/div/div/article/div[20]/div/pre/code/span[66]"}, {"position_id": 2652, "txt": "/bin/addgroup\"", "x": "/html/body/div/div/div/article/div[20]/div/pre/code/span[67]"}, {"position_id": 2654, "txt": ";}", "x": "/html/body/div/div/div/article/div[20]/div/pre/code/span[68]"}, {"position_id": 2656, "txt": "{", "x": "/html/body/div/div/div/article/div[20]/div/pre/code/span[69]"}, {"position_id": 2658, "txt": "src", "x": "/html/body/div/div/div/article/div[20]/div/pre/code/span[70]"}, {"position_id": 2660, "txt": "=", "x": "/html/body/div/div/div/article/div[20]/div/pre/code/span[71]"}, {"position_id": 2662, "txt": "\"", "x": "/html/body/div/div/div/article/div[20]/div/pre/code/span[72]"}, {"position_id": 2664, "txt": "${", "x": "/html/body/div/div/div/article/div[20]/div/pre/code/span[73]"}, {"position_id": 2666, "txt": "su", "x": "/html/body/div/div/div/article/div[20]/div/pre/code/span[74]"}, {"position_id": 2668, "txt": "}", "x": "/html/body/div/div/div/article/div[20]/div/pre/code/span[75]"}, {"position_id": 2670, "txt": "/bin/groupadd\"", "x": "/html/body/div/div/div/article/div[20]/div/pre/code/span[76]"}, {"position_id": 2672, "txt": ";}", "x": "/html/body/div/div/div/article/div[20]/div/pre/code/span[77]"}, {"position_id": 2674, "txt": "{", "x": "/html/body/div/div/div/article/div[20]/div/pre/code/span[78]"}, {"position_id": 2676, "txt": "src", "x": "/html/body/div/div/div/article/div[20]/div/pre/code/span[79]"}, {"position_id": 2678, "txt": "=", "x": "/html/body/div/div/div/article/div[20]/div/pre/code/span[80]"}, {"position_id": 2680, "txt": "\"", "x": "/html/body/div/div/div/article/div[20]/div/pre/code/span[81]"}, {"position_id": 2682, "txt": "${", "x": "/html/body/div/div/div/article/div[20]/div/pre/code/span[82]"}, {"position_id": 2684, "txt": "su", "x": "/html/body/div/div/div/article/div[20]/div/pre/code/span[83]"}, {"position_id": 2686, "txt": "}", "x": "/html/body/div/div/div/article/div[20]/div/pre/code/span[84]"}, {"position_id": 2688, "txt": "/bin/usermod\"", "x": "/html/body/div/div/div/article/div[20]/div/pre/code/span[85]"}, {"position_id": 2690, "txt": ";}", "x": "/html/body/div/div/div/article/div[20]/div/pre/code/span[86]"}, {"position_id": 2692, "txt": "{", "x": "/html/body/div/div/div/article/div[20]/div/pre/code/span[87]"}, {"position_id": 2694, "txt": "src", "x": "/html/body/div/div/div/article/div[20]/div/pre/code/span[88]"}, {"position_id": 2696, "txt": "=", "x": "/html/body/div/div/div/article/div[20]/div/pre/code/span[89]"}, {"position_id": 2698, "txt": "\"", "x": "/html/body/div/div/div/article/div[20]/div/pre/code/span[90]"}, {"position_id": 2700, "txt": "${", "x": "/html/body/div/div/div/article/div[20]/div/pre/code/span[91]"}, {"position_id": 2702, "txt": "busybox", "x": "/html/body/div/div/div/article/div[20]/div/pre/code/span[92]"}, {"position_id": 2704, "txt": "}", "x": "/html/body/div/div/div/article/div[20]/div/pre/code/span[93]"}, {"position_id": 2706, "txt": "/bin/busybox\"", "x": "/html/body/div/div/div/article/div[20]/div/pre/code/span[94]"}, {"position_id": 2708, "txt": ";}", "x": "/html/body/div/div/div/article/div[20]/div/pre/code/span[95]"}, {"position_id": 2710, "txt": "{", "x": "/html/body/div/div/div/article/div[20]/div/pre/code/span[96]"}, {"position_id": 2712, "txt": "src", "x": "/html/body/div/div/div/article/div[20]/div/pre/code/span[97]"}, {"position_id": 2714, "txt": "=", "x": "/html/body/div/div/div/article/div[20]/div/pre/code/span[98]"}, {"position_id": 2716, "txt": "\"", "x": "/html/body/div/div/div/article/div[20]/div/pre/code/span[99]"}, {"position_id": 2718, "txt": "${", "x": "/html/body/div/div/div/article/div[20]/div/pre/code/span[100]"}, {"position_id": 2720, "txt": "busybox", "x": "/html/body/div/div/div/article/div[20]/div/pre/code/span[101]"}, {"position_id": 2722, "txt": "}", "x": "/html/body/div/div/div/article/div[20]/div/pre/code/span[102]"}, {"position_id": 2724, "txt": "/bin/chmod\"", "x": "/html/body/div/div/div/article/div[20]/div/pre/code/span[103]"}, {"position_id": 2726, "txt": ";}", "x": "/html/body/div/div/div/article/div[20]/div/pre/code/span[104]"}, {"position_id": 2728, "txt": "{", "x": "/html/body/div/div/div/article/div[20]/div/pre/code/span[105]"}, {"position_id": 2730, "txt": "src", "x": "/html/body/div/div/div/article/div[20]/div/pre/code/span[106]"}, {"position_id": 2732, "txt": "=", "x": "/html/body/div/div/div/article/div[20]/div/pre/code/span[107]"}, {"position_id": 2734, "txt": "\"", "x": "/html/body/div/div/div/article/div[20]/div/pre/code/span[108]"}, {"position_id": 2736, "txt": "${", "x": "/html/body/div/div/div/article/div[20]/div/pre/code/span[109]"}, {"position_id": 2738, "txt": "busybox", "x": "/html/body/div/div/div/article/div[20]/div/pre/code/span[110]"}, {"position_id": 2740, "txt": "}", "x": "/html/body/div/div/div/article/div[20]/div/pre/code/span[111]"}, {"position_id": 2742, "txt": "/bin/cp\"", "x": "/html/body/div/div/div/article/div[20]/div/pre/code/span[112]"}, {"position_id": 2744, "txt": ";}", "x": "/html/body/div/div/div/article/div[20]/div/pre/code/span[113]"}, {"position_id": 2746, "txt": "{", "x": "/html/body/div/div/div/article/div[20]/div/pre/code/span[114]"}, {"position_id": 2748, "txt": "src", "x": "/html/body/div/div/div/article/div[20]/div/pre/code/span[115]"}, {"position_id": 2750, "txt": "=", "x": "/html/body/div/div/div/article/div[20]/div/pre/code/span[116]"}, {"position_id": 2752, "txt": "\"", "x": "/html/body/div/div/div/article/div[20]/div/pre/code/span[117]"}, {"position_id": 2754, "txt": "${", "x": "/html/body/div/div/div/article/div[20]/div/pre/code/span[118]"}, {"position_id": 2756, "txt": "busybox", "x": "/html/body/div/div/div/article/div[20]/div/pre/code/span[119]"}, {"position_id": 2758, "txt": "}", "x": "/html/body/div/div/div/article/div[20]/div/pre/code/span[120]"}, {"position_id": 2760, "txt": "/bin/cut\"", "x": "/html/body/div/div/div/article/div[20]/div/pre/code/span[121]"}, {"position_id": 2762, "txt": ";}", "x": "/html/body/div/div/div/article/div[20]/div/pre/code/span[122]"}, {"position_id": 2764, "txt": "{", "x": "/html/body/div/div/div/article/div[20]/div/pre/code/span[123]"}, {"position_id": 2766, "txt": "src", "x": "/html/body/div/div/div/article/div[20]/div/pre/code/span[124]"}, {"position_id": 2768, "txt": "=", "x": "/html/body/div/div/div/article/div[20]/div/pre/code/span[125]"}, {"position_id": 2770, "txt": "\"", "x": "/html/body/div/div/div/article/div[20]/div/pre/code/span[126]"}, {"position_id": 2772, "txt": "${", "x": "/html/body/div/div/div/article/div[20]/div/pre/code/span[127]"}, {"position_id": 2774, "txt": "busybox", "x": "/html/body/div/div/div/article/div[20]/div/pre/code/span[128]"}, {"position_id": 2776, "txt": "}", "x": "/html/body/div/div/div/article/div[20]/div/pre/code/span[129]"}, {"position_id": 2778, "txt": "/bin/getent\"", "x": "/html/body/div/div/div/article/div[20]/div/pre/code/span[130]"}, {"position_id": 2780, "txt": ";}", "x": "/html/body/div/div/div/article/div[20]/div/pre/code/span[131]"}, {"position_id": 2782, "txt": "{", "x": "/html/body/div/div/div/article/div[20]/div/pre/code/span[132]"}, {"position_id": 2784, "txt": "src", "x": "/html/body/div/div/div/article/div[20]/div/pre/code/span[133]"}, {"position_id": 2786, "txt": "=", "x": "/html/body/div/div/div/article/div[20]/div/pre/code/span[134]"}, {"position_id": 2788, "txt": "\"", "x": "/html/body/div/div/div/article/div[20]/div/pre/code/span[135]"}, {"position_id": 2790, "txt": "${", "x": "/html/body/div/div/div/article/div[20]/div/pre/code/span[136]"}, {"position_id": 2792, "txt": "busybox", "x": "/html/body/div/div/div/article/div[20]/div/pre/code/span[137]"}, {"position_id": 2794, "txt": "}", "x": "/html/body/div/div/div/article/div[20]/div/pre/code/span[138]"}, {"position_id": 2796, "txt": "/bin/head\"", "x": "/html/body/div/div/div/article/div[20]/div/pre/code/span[139]"}, {"position_id": 2798, "txt": ";}", "x": "/html/body/div/div/div/article/div[20]/div/pre/code/span[140]"}, {"position_id": 2800, "txt": "{", "x": "/html/body/div/div/div/article/div[20]/div/pre/code/span[141]"}, {"position_id": 2802, "txt": "src", "x": "/html/body/div/div/div/article/div[20]/div/pre/code/span[142]"}, {"position_id": 2804, "txt": "=", "x": "/html/body/div/div/div/article/div[20]/div/pre/code/span[143]"}, {"position_id": 2806, "txt": "\"", "x": "/html/body/div/div/div/article/div[20]/div/pre/code/span[144]"}, {"position_id": 2808, "txt": "${", "x": "/html/body/div/div/div/article/div[20]/div/pre/code/span[145]"}, {"position_id": 2810, "txt": "busybox", "x": "/html/body/div/div/div/article/div[20]/div/pre/code/span[146]"}, {"position_id": 2812, "txt": "}", "x": "/html/body/div/div/div/article/div[20]/div/pre/code/span[147]"}, {"position_id": 2814, "txt": "/bin/mktemp\"", "x": "/html/body/div/div/div/article/div[20]/div/pre/code/span[148]"}, {"position_id": 2816, "txt": ";}", "x": "/html/body/div/div/div/article/div[20]/div/pre/code/span[149]"}, {"position_id": 2818, "txt": "{", "x": "/html/body/div/div/div/article/div[20]/div/pre/code/span[150]"}, {"position_id": 2820, "txt": "src", "x": "/html/body/div/div/div/article/div[20]/div/pre/code/span[151]"}, {"position_id": 2822, "txt": "=", "x": "/html/body/div/div/div/article/div[20]/div/pre/code/span[152]"}, {"position_id": 2824, "txt": "\"", "x": "/html/body/div/div/div/article/div[20]/div/pre/code/span[153]"}, {"position_id": 2826, "txt": "${", "x": "/html/body/div/div/div/article/div[20]/div/pre/code/span[154]"}, {"position_id": 2828, "txt": "busybox", "x": "/html/body/div/div/div/article/div[20]/div/pre/code/span[155]"}, {"position_id": 2830, "txt": "}", "x": "/html/body/div/div/div/article/div[20]/div/pre/code/span[156]"}, {"position_id": 2832, "txt": "/bin/rm\"", "x": "/html/body/div/div/div/article/div[20]/div/pre/code/span[157]"}, {"position_id": 2834, "txt": ";}", "x": "/html/body/div/div/div/article/div[20]/div/pre/code/span[158]"}, {"position_id": 2836, "txt": "{", "x": "/html/body/div/div/div/article/div[20]/div/pre/code/span[159]"}, {"position_id": 2838, "txt": "src", "x": "/html/body/div/div/div/article/div[20]/div/pre/code/span[160]"}, {"position_id": 2840, "txt": "=", "x": "/html/body/div/div/div/article/div[20]/div/pre/code/span[161]"}, {"position_id": 2842, "txt": "\"", "x": "/html/body/div/div/div/article/div[20]/div/pre/code/span[162]"}, {"position_id": 2844, "txt": "${", "x": "/html/body/div/div/div/article/div[20]/div/pre/code/span[163]"}, {"position_id": 2846, "txt": "busybox", "x": "/html/body/div/div/div/article/div[20]/div/pre/code/span[164]"}, {"position_id": 2848, "txt": "}", "x": "/html/body/div/div/div/article/div[20]/div/pre/code/span[165]"}, {"position_id": 2850, "txt": "/bin/sed\"", "x": "/html/body/div/div/div/article/div[20]/div/pre/code/span[166]"}, {"position_id": 2852, "txt": ";}", "x": "/html/body/div/div/div/article/div[20]/div/pre/code/span[167]"}, {"position_id": 2854, "txt": "{", "x": "/html/body/div/div/div/article/div[20]/div/pre/code/span[168]"}, {"position_id": 2856, "txt": "src", "x": "/html/body/div/div/div/article/div[20]/div/pre/code/span[169]"}, {"position_id": 2858, "txt": "=", "x": "/html/body/div/div/div/article/div[20]/div/pre/code/span[170]"}, {"position_id": 2860, "txt": "\"", "x": "/html/body/div/div/div/article/div[20]/div/pre/code/span[171]"}, {"position_id": 2862, "txt": "${", "x": "/html/body/div/div/div/article/div[20]/div/pre/code/span[172]"}, {"position_id": 2864, "txt": "busybox", "x": "/html/body/div/div/div/article/div[20]/div/pre/code/span[173]"}, {"position_id": 2866, "txt": "}", "x": "/html/body/div/div/div/article/div[20]/div/pre/code/span[174]"}, {"position_id": 2868, "txt": "/bin/tail\"", "x": "/html/body/div/div/div/article/div[20]/div/pre/code/span[175]"}, {"position_id": 2870, "txt": ";}", "x": "/html/body/div/div/div/article/div[20]/div/pre/code/span[176]"}, {"position_id": 2872, "txt": "{", "x": "/html/body/div/div/div/article/div[20]/div/pre/code/span[177]"}, {"position_id": 2874, "txt": "src", "x": "/html/body/div/div/div/article/div[20]/div/pre/code/span[178]"}, {"position_id": 2876, "txt": "=", "x": "/html/body/div/div/div/article/div[20]/div/pre/code/span[179]"}, {"position_id": 2878, "txt": "\"", "x": "/html/body/div/div/div/article/div[20]/div/pre/code/span[180]"}, {"position_id": 2880, "txt": "${", "x": "/html/body/div/div/div/article/div[20]/div/pre/code/span[181]"}, {"position_id": 2882, "txt": "busybox", "x": "/html/body/div/div/div/article/div[20]/div/pre/code/span[182]"}, {"position_id": 2884, "txt": "}", "x": "/html/body/div/div/div/article/div[20]/div/pre/code/span[183]"}, {"position_id": 2886, "txt": "/bin/uname\"", "x": "/html/body/div/div/div/article/div[20]/div/pre/code/span[184]"}, {"position_id": 2888, "txt": ";}", "x": "/html/body/div/div/div/article/div[20]/div/pre/code/span[185]"}, {"position_id": 2890, "txt": "{", "x": "/html/body/div/div/div/article/div[20]/div/pre/code/span[186]"}, {"position_id": 2892, "txt": "src", "x": "/html/body/div/div/div/article/div[20]/div/pre/code/span[187]"}, {"position_id": 2894, "txt": "=", "x": "/html/body/div/div/div/article/div[20]/div/pre/code/span[188]"}, {"position_id": 2896, "txt": "\"", "x": "/html/body/div/div/div/article/div[20]/div/pre/code/span[189]"}, {"position_id": 2898, "txt": "${", "x": "/html/body/div/div/div/article/div[20]/div/pre/code/span[190]"}, {"position_id": 2900, "txt": "busybox", "x": "/html/body/div/div/div/article/div[20]/div/pre/code/span[191]"}, {"position_id": 2902, "txt": "}", "x": "/html/body/div/div/div/article/div[20]/div/pre/code/span[192]"}, {"position_id": 2904, "txt": "/bin/which\"", "x": "/html/body/div/div/div/article/div[20]/div/pre/code/span[193]"}, {"position_id": 2906, "txt": ";}", "x": "/html/body/div/div/div/article/div[20]/div/pre/code/span[194]"}, {"position_id": 2908, "txt": "];", "x": "/html/body/div/div/div/article/div[20]/div/pre/code/span[195]"}, {"position_id": 2910, "txt": "};", "x": "/html/body/div/div/div/article/div[20]/div/pre/code/span[196]"}]}, "tags": [], "label": "code", "web_segment_id": 108, "global_sentence_id": 399, "edu_l1_label": "EDU_O"}, {"txt": "The rule of thumbs for digging the rabbit holes", "language": "english", "position": {"atoms": [{"position_id": 2912, "txt": "The rule of thumbs for digging the rabbit holes", "x": "/html/body/div/div/div/article/h2[8]"}]}, "tags": ["h2"], "label": "title1", "web_segment_id": 109, "global_sentence_id": 400, "edu_l1_label": "BOS"}, {"txt": "Now, you’ve seen how I dug many rabbit holes in the past week.", "language": "english", "position": {"atoms": [{"position_id": 2914, "txt": "Now, you’ve seen how I dug many rabbit holes in the past week.", "x": "/html/body/div/div/div/article/p[61]"}]}, "tags": [], "label": "content", "web_segment_id": 110, "global_sentence_id": 401, "edu_l1_label": "IOS"}, {"txt": " I tried to summarize my approach to digging the chain of rabbit holes:", "language": "english", "position": {"atoms": [{"position_id": 2915, "txt": " I tried to summarize my approach to digging the chain of rabbit holes:", "x": "/html/body/div/div/div/article/p[61]"}]}, "tags": [], "label": "content", "web_segment_id": 110, "global_sentence_id": 402, "edu_l1_label": "IOS"}, {"txt": "Consider the productivity impact.", "language": "english", "position": {"atoms": [{"position_id": 2917, "txt": "Consider the productivity impact.", "x": "/html/body/div/div/div/article/ul/li[1]"}]}, "tags": ["li"], "label": "content", "web_segment_id": 111, "global_sentence_id": 403, "edu_l1_label": "IOS"}, {"txt": " If it provides a great positive impact within a manageable timeframe, do it as soon as possible (nix-playground as a great example)", "language": "english", "position": {"atoms": [{"position_id": 2918, "txt": " If it provides a great positive impact within a manageable timeframe, do it as soon as possible (nix-playground as a great example)", "x": "/html/body/div/div/div/article/ul/li[1]"}]}, "tags": ["li"], "label": "content", "web_segment_id": 111, "global_sentence_id": 404, "edu_l1_label": "IOS"}, {"txt": "Not all rabbit holes are worth digging; consider the cost-effect ratio.", "language": "english", "position": {"atoms": [{"position_id": 2920, "txt": "Not all rabbit holes are worth digging; consider the cost-effect ratio.", "x": "/html/body/div/div/div/article/ul/li[2]"}]}, "tags": ["li"], "label": "content", "web_segment_id": 112, "global_sentence_id": 405, "edu_l1_label": "IOS"}, {"txt": " For example, I won’t jump into learning PTX and optimizing my MAZE code before it runs on a huge scale.", "language": "english", "position": {"atoms": [{"position_id": 2921, "txt": " For example, I won’t jump into learning PTX and optimizing my MAZE code before it runs on a huge scale.", "x": "/html/body/div/div/div/article/ul/li[2]"}]}, "tags": ["li"], "label": "content", "web_segment_id": 112, "global_sentence_id": 406, "edu_l1_label": "IOS"}, {"txt": "Evaluate the timeframe of digging a hole to carefully decide whether to go down a hole.", "language": "english", "position": {"atoms": [{"position_id": 2923, "txt": "Evaluate the timeframe of digging a hole to carefully decide whether to go down a hole.", "x": "/html/body/div/div/div/article/ul/li[3]"}]}, "tags": ["li"], "label": "content", "web_segment_id": 113, "global_sentence_id": 407, "edu_l1_label": "IOS"}, {"txt": " Sometimes, it’s hard to tell from the surface; a quick prototype and probing could help determine how hard it is.", "language": "english", "position": {"atoms": [{"position_id": 2924, "txt": " Sometimes, it’s hard to tell from the surface; a quick prototype and probing could help determine how hard it is.", "x": "/html/body/div/div/div/article/ul/li[3]"}]}, "tags": ["li"], "label": "content", "web_segment_id": 113, "global_sentence_id": 408, "edu_l1_label": "IOS"}, {"txt": "Enjoy digging rabbit holes because it’s the best way to learn new things", "language": "english", "position": {"atoms": [{"position_id": 2926, "txt": "Enjoy digging rabbit holes because it’s the best way to learn new things", "x": "/html/body/div/div/div/article/ul/li[4]"}]}, "tags": ["li"], "label": "content", "web_segment_id": 114, "global_sentence_id": 409, "edu_l1_label": "IOS"}, {"txt": "Final thoughts", "language": "english", "position": {"atoms": [{"position_id": 2928, "txt": "Final thoughts", "x": "/html/body/div/div/div/article/h1"}]}, "tags": ["h1"], "label": "title1", "web_segment_id": 115, "global_sentence_id": 410, "edu_l1_label": "BOS"}, {"txt": "I intentionally left out many details from this article because otherwise, it would be too long.", "language": "english", "position": {"atoms": [{"position_id": 2930, "txt": "I intentionally left out many details from this article because otherwise, it would be too long.", "x": "/html/body/div/div/div/article/p[62]"}]}, "tags": [], "label": "content", "web_segment_id": 116, "global_sentence_id": 411, "edu_l1_label": "IOS"}, {"txt": " If anyone is interested in learning more about running Nvidia CDI on a local Kubernetes cluster, please leave a comment and let me know.", "language": "english", "position": {"atoms": [{"position_id": 2931, "txt": " If anyone is interested in learning more about running Nvidia CDI on a local Kubernetes cluster, please leave a comment and let me know.", "x": "/html/body/div/div/div/article/p[62]"}]}, "tags": [], "label": "content", "web_segment_id": 116, "global_sentence_id": 412, "edu_l1_label": "IOS"}, {"txt": " I am considering open-sourcing some of my work to make it much easier for anyone to have a local Kubernetes cluster enabled with Nvidia CDI.", "language": "english", "position": {"atoms": [{"position_id": 2932, "txt": " I am considering open-sourcing some of my work to make it much easier for anyone to have a local Kubernetes cluster enabled with Nvidia CDI.", "x": "/html/body/div/div/div/article/p[62]"}]}, "tags": [], "label": "content", "web_segment_id": 116, "global_sentence_id": 413, "edu_l1_label": "IOS"}, {"txt": "This deviates a bit from building MAZE, but the outcome is great!", "language": "english", "position": {"atoms": [{"position_id": 2934, "txt": "This deviates a bit from building MAZE, but the outcome is great!", "x": "/html/body/div/div/div/article/p[63]"}]}, "tags": [], "label": "content", "web_segment_id": 117, "global_sentence_id": 414, "edu_l1_label": "IOS"}, {"txt": " Now I have an infra for running MAZE on a bigger scale—well, at least slightly bigger on the scale I can afford 😅 There are still other rabbit holes, like power consumption.", "language": "english", "position": {"atoms": [{"position_id": 2935, "txt": " Now I have an infra for running MAZE on a bigger scale—well, at least slightly bigger on the scale I can afford 😅 There are still other rabbit holes, like power consumption.", "x": "/html/body/div/div/div/article/p[63]"}]}, "tags": [], "label": "content", "web_segment_id": 117, "global_sentence_id": 415, "edu_l1_label": "IOS"}, {"txt": " With the constraints, I can only try to be creative.", "language": "english", "position": {"atoms": [{"position_id": 2936, "txt": " With the constraints, I can only try to be creative.", "x": "/html/body/div/div/div/article/p[63]"}]}, "tags": [], "label": "content", "web_segment_id": 117, "global_sentence_id": 416, "edu_l1_label": "IOS"}, {"txt": "Next, I will go back to improving MAZE and soon publish the third MAZE article.", "language": "english", "position": {"atoms": [{"position_id": 2938, "txt": "Next, I will go back to improving MAZE and soon publish the third MAZE article.", "x": "/html/body/div/div/div/article/p[64]"}]}, "tags": [], "label": "content", "web_segment_id": 118, "global_sentence_id": 417, "edu_l1_label": "IOS"}, {"txt": " I have been running the experiment for the past week, and it has already accumulated some data with updated MAZE code, including mutations and some other good stuff.", "language": "english", "position": {"atoms": [{"position_id": 2939, "txt": " I have been running the experiment for the past week, and it has already accumulated some data with updated MAZE code, including mutations and some other good stuff.", "x": "/html/body/div/div/div/article/p[64]"}]}, "tags": [], "label": "content", "web_segment_id": 118, "global_sentence_id": 418, "edu_l1_label": "IOS"}, {"txt": " Likewise, I will publish the data along with the next article.", "language": "english", "position": {"atoms": [{"position_id": 2940, "txt": " Likewise, I will publish the data along with the next article.", "x": "/html/body/div/div/div/article/p[64]"}]}, "tags": [], "label": "content", "web_segment_id": 118, "global_sentence_id": 419, "edu_l1_label": "IOS"}, {"txt": " I am still thinking about the next focus of the following research.", "language": "english", "position": {"atoms": [{"position_id": 2941, "txt": " I am still thinking about the next focus of the following research.", "x": "/html/body/div/div/div/article/p[64]"}]}, "tags": [], "label": "content", "web_segment_id": 118, "global_sentence_id": 420, "edu_l1_label": "IOS"}, {"txt": " I have some interesting ideas for eliminating backpropagation with the MAZE approach.", "language": "english", "position": {"atoms": [{"position_id": 2942, "txt": " I have some interesting ideas for eliminating backpropagation with the MAZE approach.", "x": "/html/body/div/div/div/article/p[64]"}]}, "tags": [], "label": "content", "web_segment_id": 118, "global_sentence_id": 421, "edu_l1_label": "IOS"}, {"txt": " I am also thinking that maybe it’s time to introduce more freedom (more operations) to MAZE’s neuron network so that I can break the accuracy barrier it’s facing.", "language": "english", "position": {"atoms": [{"position_id": 2943, "txt": " I am also thinking that maybe it’s time to introduce more freedom (more operations) to MAZE’s neuron network so that I can break the accuracy barrier it’s facing.", "x": "/html/body/div/div/div/article/p[64]"}]}, "tags": [], "label": "content", "web_segment_id": 118, "global_sentence_id": 422, "edu_l1_label": "IOS"}, {"txt": " Anyway, we will see.", "language": "english", "position": {"atoms": [{"position_id": 2944, "txt": " Anyway, we will see.", "x": "/html/body/div/div/div/article/p[64]"}]}, "tags": [], "label": "content", "web_segment_id": 118, "global_sentence_id": 423, "edu_l1_label": "IOS"}, {"txt": "Regardless, thanks for reading.", "language": "english", "position": {"atoms": [{"position_id": 2946, "txt": "Regardless, thanks for reading.", "x": "/html/body/div/div/div/article/p[65]"}]}, "tags": [], "label": "content", "web_segment_id": 119, "global_sentence_id": 424, "edu_l1_label": "IOS"}, {"txt": " I hope you like my journey of digging the rabbit hole and sharing knowledge about running Nvidia GPU on Kubernetes.", "language": "english", "position": {"atoms": [{"position_id": 2947, "txt": " I hope you like my journey of digging the rabbit hole and sharing knowledge about running Nvidia GPU on Kubernetes.", "x": "/html/body/div/div/div/article/p[65]"}]}, "tags": [], "label": "content", "web_segment_id": 119, "global_sentence_id": 425, "edu_l1_label": "IOS"}, {"txt": " Please help me share this article if you find it interesting.", "language": "english", "position": {"atoms": [{"position_id": 2948, "txt": " Please help me share this article if you find it interesting.", "x": "/html/body/div/div/div/article/p[65]"}]}, "tags": [], "label": "content", "web_segment_id": 119, "global_sentence_id": 426, "edu_l1_label": "IOS"}, {"txt": " Stay tuned because we will have tons of fun on the journey of learning machine learning by building a novel project like MAZE!", "language": "english", "position": {"atoms": [{"position_id": 2949, "txt": " Stay tuned because we will have tons of fun on the journey of learning machine learning by building a novel project like MAZE!", "x": "/html/body/div/div/div/article/p[65]"}]}, "tags": [], "label": "content", "web_segment_id": 119, "global_sentence_id": 427, "edu_l1_label": "IOS"}, {"txt": " 😄", "language": "english", "position": {"atoms": [{"position_id": 2950, "txt": " 😄", "x": "/html/body/div/div/div/article/p[65]"}]}, "tags": [], "label": "content", "web_segment_id": 119, "global_sentence_id": 428, "edu_l1_label": "IOS"}], "type": "WEB"}
# Nvidia GPU on bare metal NixOS Kubernetes cluster explained
## Chain of rabbit holes
## Running Kubernetes with NixOS
## Kubernetes with Nvidia GPU CDI plugin explained
### Container Runtime Interface
### nvidia-container-toolkit
### libnvidia-container
### Container Device Interface
### k8s-device-plugin
## Dig this hole first - build nix-playground for hacking and patching any source code easily
## Hunting the k8s-device-plugin crashing bug
## The missing RuntimeClass
## Yet another rabbit hole - PyCharm cannot open my WSL NixOS anymore
## The rule of thumbs for digging the rabbit holes
## Final thoughts
