1 00:00:00,000 --> 00:00:18,420 *36C3 preroll music* 2 00:00:18,420 --> 00:00:24,530 Herald-Angel: This Talk will be about… I have to read this. Mathematical diseases 3 00:00:24,530 --> 00:00:28,600 in climate models and how to cure them. And I don't have the slightest idea what 4 00:00:28,600 --> 00:00:33,850 these two guys are talking about now. And when I asked them, they said, just tell 5 00:00:33,850 --> 00:00:40,039 the people it's about next generation climate models and how to build them. 6 00:00:40,039 --> 00:00:45,789 Which is cool. Throw that on Twitter. Please welcome Ali Ramadhan and Valentin 7 00:00:45,789 --> 00:00:47,229 Churavy. 8 00:00:47,229 --> 00:00:55,239 *applause* 9 00:00:55,239 --> 00:00:58,509 Ali Ramadhan: Can you guys hear us? Is this OK? 10 00:00:58,509 --> 00:01:00,210 Valentin Churavy: I'll stand back... Ramadhan: I'll stand back a little bit. 11 00:01:00,210 --> 00:01:05,030 OK, cool. Thank you. So if you guys saw the last talk by karlab... karlabyrinth or 12 00:01:05,030 --> 00:01:08,530 something. So we're kind of expanding on her talk a little bit. So she talked a lot 13 00:01:08,530 --> 00:01:11,209 about kind of uncertainties… *audio feedback from microphone* 14 00:01:11,209 --> 00:01:15,670 uncertainties in climate models. And one point that she did make was that most of 15 00:01:15,670 --> 00:01:18,410 the uncertainty actually comes from humans. But there's a really huge 16 00:01:18,410 --> 00:01:22,070 uncertainty that also comes from… comes from the models. So we're talking more 17 00:01:22,070 --> 00:01:26,170 about the model uncertainties, which is kind of uncertainties because of unknown 18 00:01:26,170 --> 00:01:31,590 or missing physics, kind of how to cure them. So it'll be kind of a weird talk. So 19 00:01:31,590 --> 00:01:35,119 I'll talk a little bit more about the climate modeling part and then kind of how 20 00:01:35,119 --> 00:01:39,840 to cure them involves using new programing languages. And that's where Valentin will 21 00:01:39,840 --> 00:01:44,810 talk about Julia. So we'll kind of just start with maybe just giving kind of an 22 00:01:44,810 --> 00:01:50,080 idea of why it's so hard to model the climate. So if you've… maybe you've seen 23 00:01:50,080 --> 00:01:54,550 images like this a lot where it's like a… it's a satellite image basically of 24 00:01:54,550 --> 00:01:58,530 clouds. It's used for like weather forecasting. But you can immediately see 25 00:01:58,530 --> 00:02:02,320 there's lots of, you know, lots of really small clouds. So basically, if you want to 26 00:02:02,320 --> 00:02:06,250 build a climate model, you've got to be able to resolve all the physics in these 27 00:02:06,250 --> 00:02:10,770 clouds. So you can actually zoom in a lot. And see the clouds look pretty big over 28 00:02:10,770 --> 00:02:15,140 here. But if you zoom in on kind of Central America, then you see even smaller 29 00:02:15,140 --> 00:02:21,509 clouds. And if you zoom in even more so, so you zoom in on the Yucatan Peninsula, 30 00:02:21,509 --> 00:02:24,780 then you can see the clouds are really, really small. So you're… there are maybe 31 00:02:24,780 --> 00:02:28,980 five smaller clouds, some some of the clouds are, you know, a hundred meters or 32 00:02:28,980 --> 00:02:33,709 something. And as the last talk kind of suggests that most climate models are… 33 00:02:33,709 --> 00:02:38,990 they resolve things of, you know, up to 50 kilometers. So anything smaller than 50 34 00:02:38,990 --> 00:02:43,670 kilometers, the climate model can't really see. So you have to kind of take that… it 35 00:02:43,670 --> 00:02:46,290 kind of has to account for that because clouds are important, and if you have more 36 00:02:46,290 --> 00:02:51,779 clouds, then that reflects some of the heat out. So maybe you cool. But it also 37 00:02:51,779 --> 00:02:55,569 traps more of the heat in so maybe you warm. And if you have more clouds, maybe 38 00:02:55,569 --> 00:03:00,130 you warm more. But if you have less clouds, maybe you warm even more. So it's 39 00:03:00,130 --> 00:03:04,549 kind of unsure. We actually don't know if clouds will make the climate warmer or if 40 00:03:04,549 --> 00:03:07,760 they'll make the climate cooler. So it's important for your climate models to kind 41 00:03:07,760 --> 00:03:13,200 of resolve or see these little clouds. So kind of where the mathematical disease 42 00:03:13,200 --> 00:03:16,940 comes in, is that you don't… we don't know what equation to solve. We don't know 43 00:03:16,940 --> 00:03:21,399 exactly what physics to solve, to see, to kind of resolve the effect of these little 44 00:03:21,399 --> 00:03:25,120 clouds. So it's kind of the the mathematical disease. We don't know how to 45 00:03:25,120 --> 00:03:29,939 do it. So you instead use a… well, it's called a parametrization, which is the 46 00:03:29,939 --> 00:03:33,120 mathematical disease. So in the atmosphere, the big mathematical disease 47 00:03:33,120 --> 00:03:40,240 is clouds. But if you look at the ocean, you kind of get a similar… You have also 48 00:03:40,240 --> 00:03:43,719 similar mathematical diseases. So if you for example, this is model output. We 49 00:03:43,719 --> 00:03:48,830 don't have good satellite imagery of the oceans. So if you if you look at, for 50 00:03:48,830 --> 00:03:53,400 example, model output from an ocean model, high resolution ocean model here, it's 51 00:03:53,400 --> 00:03:58,159 kind of centered on the Pacific. So you can kind of see Japan and China and the 52 00:03:58,159 --> 00:04:03,060 white kind of lines. Those are streamlines or that the lines tell you where the water 53 00:04:03,060 --> 00:04:07,019 is going. So you could see a lot of kind of straight lines. You see this curve here 54 00:04:07,019 --> 00:04:13,669 current off of Japan, but you see lots of circles. So the circles are these eddies 55 00:04:13,669 --> 00:04:17,910 and they're kind of the turbulence of the ocean. They move, they kind of stir and 56 00:04:17,910 --> 00:04:23,740 mix and transport a lot of salt or heat or carbon or nutrients or… you know, marine 57 00:04:23,740 --> 00:04:28,270 life or anything. It's the main way the ocean kind of moves heat from the equator 58 00:04:28,270 --> 00:04:32,970 to the pole. It kind of stirs things around. So they're really important for 59 00:04:32,970 --> 00:04:38,480 kind of how carbon moves in the ocean, for how the ocean heats up. And here they look 60 00:04:38,480 --> 00:04:42,259 pretty big. But again, you can zoom in and you'll see lots of small scale structures. 61 00:04:42,259 --> 00:04:46,500 So we're going to switch to a different model output and different colors. So here 62 00:04:46,500 --> 00:04:51,260 here's kind of the same area. So you see Japan in the top left. But what's being 63 00:04:51,260 --> 00:04:55,720 plotted is vorticity. So you have to know what that is. It's kind of a measure of 64 00:04:55,720 --> 00:05:00,030 how much the fluid or the water is spinning. But the point is that you have 65 00:05:00,030 --> 00:05:05,340 lots of structure. So there's lots of, you know, big circles, but there also lots of 66 00:05:05,340 --> 00:05:10,070 really little circles. And again, your climate model can only see something like 67 00:05:10,070 --> 00:05:14,919 50 kilometers or 100 kilometers. But as you can see here, there's lots of stuff 68 00:05:14,919 --> 00:05:18,860 that's much smaller than a hundred kilometers. So if you superimpose kind of 69 00:05:18,860 --> 00:05:23,880 this this grid, maybe that's your climate model grid. And, you know, basically for 70 00:05:23,880 --> 00:05:27,840 the climate model, every one of these boxes is like one number. So you can't 71 00:05:27,840 --> 00:05:31,880 really see anything smaller than that. But there's important dynamics and physics 72 00:05:31,880 --> 00:05:35,380 that happens in like 10 kilometers, which is a lot smaller than what the climate 73 00:05:35,380 --> 00:05:39,470 model can see. And there's even important physics that happens that like 100 meters 74 00:05:39,470 --> 00:05:44,290 or 200 meters. So if you want if you want to, you know, what the climate will look 75 00:05:44,290 --> 00:05:50,909 like, we need to… we need to know about the physics that happens at 200 meters. So 76 00:05:50,909 --> 00:05:55,020 to give an example of some of the physics that happens at 10 kilometers, here's kind 77 00:05:55,020 --> 00:06:01,040 of a little animation where this kind of explains why you get all these eddies or 78 00:06:01,040 --> 00:06:05,530 all the circles in the ocean. So a lot of times you have, say, hot water, say, in 79 00:06:05,530 --> 00:06:10,520 the north. So the hot water here is all in orange or yellow and you have a lot of 80 00:06:10,520 --> 00:06:15,340 cold water. So the cold water is in the south and it's purple. And then once this… 81 00:06:15,340 --> 00:06:20,099 once you add rotation, you end up with these eddies because what the hot water 82 00:06:20,099 --> 00:06:24,350 wants to do, the hot water is lighter, it's less dense. So it actually wants to 83 00:06:24,350 --> 00:06:28,810 go on top of the cold water. So usually have cold at the bottom, hot at the top. 84 00:06:28,810 --> 00:06:34,120 So you have heavy at the bottom and light at the top. So when you add… without 85 00:06:34,120 --> 00:06:38,050 rotation, the hot water will just go on top of the cold water. But when you have 86 00:06:38,050 --> 00:06:42,540 rotation, you end up… it kind of wants to tip over. But it's also rotating. So you 87 00:06:42,540 --> 00:06:47,479 kind of get this beautiful swirling patterns and these are kind of the same 88 00:06:47,479 --> 00:06:52,380 circular eddies that you see in the real ocean. But this model here is like two 89 00:06:52,380 --> 00:06:55,139 hundred and fifty kilometers by five hundred kilometers and it's like one 90 00:06:55,139 --> 00:06:59,710 kilometer deep. So you need a lot of resolution to be able to resolve this 91 00:06:59,710 --> 00:07:04,750 stuff, but not… your climate model doesn't have that much resolution. So some of the 92 00:07:04,750 --> 00:07:08,449 features here, like the sharp prints between the cold and the hot water, your 93 00:07:08,449 --> 00:07:12,879 climate model might not see that. So maybe if you if you don't resolve this properly, 94 00:07:12,879 --> 00:07:17,270 you get the mixing rate wrong or maybe that the ocean is the wrong temperature or 95 00:07:17,270 --> 00:07:22,039 something. So it's kind of important to resolve this stuff. Another one, the color 96 00:07:22,039 --> 00:07:26,170 scheme here is really bad. *laughs* I'm sorry, but another one, for example, is 97 00:07:26,170 --> 00:07:32,389 here. Everything's under 100 meter, so it's a cube of 100 meters on each side and 98 00:07:32,389 --> 00:07:37,340 you're starting with 20 degrees Celsius water at the top. You have 19 degrees 99 00:07:37,340 --> 00:07:41,370 Celsius water at the bottom initially. So it's kind of you're… as you go deeper in 100 00:07:41,370 --> 00:07:46,270 the ocean, the water gets colder. And then if you can imagine, the ocean kind of at 101 00:07:46,270 --> 00:07:50,669 night, it's kind of cold. So the top is being cooled and you end up with cold 102 00:07:50,669 --> 00:07:54,639 water on the top. The cold water wants to be at the bottom. So it ends up sinking and you 103 00:07:54,639 --> 00:07:59,030 get all this convection going on. So this is happening at a lot of places in the 104 00:07:59,030 --> 00:08:03,650 ocean. You get a lot of mixing at the top. You get this kind of layer at the top of 105 00:08:03,650 --> 00:08:08,090 the ocean. It's kind of constant color, constant temperature. So this mix layer is 106 00:08:08,090 --> 00:08:12,240 important for the ocean. So knowing how deep that mix layer is and knowing how 107 00:08:12,240 --> 00:08:16,000 much of the water is being mixed is also important for for climate. But as you can 108 00:08:16,000 --> 00:08:19,860 imagine, you know, if this happens on very small scales. So you're climate model has 109 00:08:19,860 --> 00:08:24,870 to know something about what's happening at this scale. So this isn't I guess the 110 00:08:24,870 --> 00:08:28,921 mathematical diseases in the ocean is, the climate model cannot see this, so it has 111 00:08:28,921 --> 00:08:32,970 to do something else that's maybe unphysical to resolve this stuff. And 112 00:08:32,970 --> 00:08:38,420 that's a mathematical disease, I guess. Aside from the ocean and the atmosphere. 113 00:08:38,420 --> 00:08:41,820 You also have the same problem with sea ice. So this is kind of just a satellite 114 00:08:41,820 --> 00:08:47,010 picture of where sea ice is forming off the coast of Antarctica. So you get winds 115 00:08:47,010 --> 00:08:48,680 that kind of come off the continent and they're 116 00:08:48,680 --> 00:08:51,080 kind of blowing all the ice that's beeing formed 117 00:08:51,080 --> 00:08:53,900 away. So you get all these little lines and streaks and they kind of 118 00:08:53,900 --> 00:08:58,390 merge into sea ice. But in this whole picture is like 20 kilometers. So the 119 00:08:58,390 --> 00:09:01,950 climate model doesn't see this, but somehow it has to represent all the 120 00:09:01,950 --> 00:09:08,561 physics. And you have kind of similar things happening with soil moisture, land 121 00:09:08,561 --> 00:09:16,240 and dynamic vegetation, aerosols. So, you know, these are kind of three places with 122 00:09:16,240 --> 00:09:21,270 pretty pictures. But see, if you look at the atmosphere, so it's not just clouds. 123 00:09:21,270 --> 00:09:27,350 You also have aerosols, which are like little particles, or sulfates that are 124 00:09:27,350 --> 00:09:31,680 important for kind of cloud formation and maybe atmospheric chemistry. But again, we 125 00:09:31,680 --> 00:09:34,700 don't fully understand the physics of these aerosols. So again, you have to kind 126 00:09:34,700 --> 00:09:40,270 of parametrize them. Same thing with kind of convictions. You maybe your climate 127 00:09:40,270 --> 00:09:44,160 model doesn't resolve all the very deep convection in the atmosphere so as to get 128 00:09:44,160 --> 00:09:48,190 all sides to parametrize it. So I guess you have many kind of mathematical 129 00:09:48,190 --> 00:09:51,291 diseases in the atmosphere. So I'm not expecting you to understand everything in 130 00:09:51,291 --> 00:09:55,110 this in this picture. But the idea is: The atmosphere is complicated. There's no way 131 00:09:55,110 --> 00:09:59,740 a climate model is going to kind of, you know, figure all this out by itself. And 132 00:09:59,740 --> 00:10:04,600 again, you could you could do something similar for the ocean. So we can just show 133 00:10:04,600 --> 00:10:07,370 an image for like two little parts of these. But the point is, you know, the 134 00:10:07,370 --> 00:10:11,490 ocean is not kind of just a bucket of water standing there. So there's lots of 135 00:10:11,490 --> 00:10:15,180 stuff happening deep inside the ocean. And some of it, we think is important for 136 00:10:15,180 --> 00:10:19,180 climate. Some of it we don't know. Some might not be important. But again, a lot 137 00:10:19,180 --> 00:10:25,430 of this happens on very small spatial scales. So we don't know or the climate 138 00:10:25,430 --> 00:10:30,441 model can't always resolve all this stuff. And again, same thing with kind of sea 139 00:10:30,441 --> 00:10:34,370 ice. Lots of small scale stuff is important for sea ice. And I think one 140 00:10:34,370 --> 00:10:37,770 person asked about kind of tipping points and there's kind of two with like sea ice 141 00:10:37,770 --> 00:10:43,880 that are pretty important. One of them is this CSL biofeedback. So if you have sea 142 00:10:43,880 --> 00:10:48,630 ice that melts. Now you have more ocean and the ocean can absorb more heat. But 143 00:10:48,630 --> 00:10:51,980 now the earth is warmer, so it melts more sea ice. So as soon as you kind of start 144 00:10:51,980 --> 00:10:55,670 melting sea ice, maybe you melt even more sea ice and eventually you reach an earth 145 00:10:55,670 --> 00:11:00,020 with no sea ice. So there's kind of research into that stuff going on, but 146 00:11:00,020 --> 00:11:04,410 it's a possible tipping point. Another one is this kind of marine ice sheet, 147 00:11:04,410 --> 00:11:08,970 stability, instability at the bottom of the ice shelf. So if you start melting 148 00:11:08,970 --> 00:11:13,500 water, if you start melting ice from the bottom of the ice shelf, then we create 149 00:11:13,500 --> 00:11:17,530 kind of a larger area for more ice to melt. So maybe once you start melting and 150 00:11:17,530 --> 00:11:20,940 increasing sea level, you just keep melting more and more and increasing sea 151 00:11:20,940 --> 00:11:27,300 level even more. But again, it's kind of hard to quantify these things on like 50 152 00:11:27,300 --> 00:11:34,910 or 100 year timescales because it all happens on very small scales. So yeah, the 153 00:11:34,910 --> 00:11:39,760 point is there's lots of these kind of parametrizations or mathematical diseases. 154 00:11:39,760 --> 00:11:43,800 And once you start adding them all up, you end up with lots and lots of kind of 155 00:11:43,800 --> 00:11:48,180 parameters. So this is a really boring table. But the point is, so this is like 156 00:11:48,180 --> 00:11:53,110 one parametrization for like vertical mixing in the ocean. It's basically the 157 00:11:53,110 --> 00:11:57,140 process that I showed the rainbow color movie about to see a climate model for 158 00:11:57,140 --> 00:12:02,060 that. I'm trying to kind of parametrize that, physics might have like 20 159 00:12:02,060 --> 00:12:05,860 parameters. And, you know, some of them are crazy like a surface layer fractional 160 00:12:05,860 --> 00:12:10,120 like zero point one or something. And usually they keep the same constants for 161 00:12:10,120 --> 00:12:14,620 all these values. Usually it's like someone in like 1994 came up with these 20 162 00:12:14,620 --> 00:12:18,730 numbers and now we all use the same 20 numbers. But you know, maybe they're 163 00:12:18,730 --> 00:12:21,850 different. And like the Pacific or the Atlantic or like maybe they're different 164 00:12:21,850 --> 00:12:25,740 when it's summer and winter and the problem is, there's many of these 165 00:12:25,740 --> 00:12:28,920 parametrizations. So you know here's like 20 parameters, but then you have a lot 166 00:12:28,920 --> 00:12:32,430 more for clouds. You have a lot more sea ice. We add them all up. Suddenly you have 167 00:12:32,430 --> 00:12:38,050 like 100, maybe up to a thousand kind of tunable parameters. Kind of going back to 168 00:12:38,050 --> 00:12:42,700 this plot that was shown at the last talk. You can see kind of the all the models 169 00:12:42,700 --> 00:12:47,880 kind of agree really well from like 1850 to 2000, because they're all kind of being 170 00:12:47,880 --> 00:12:50,950 they all have different kind of parameters, but they all get kind of tuned 171 00:12:50,950 --> 00:12:54,890 or optimized. So they get the 20th century. Correct. So they get the black 172 00:12:54,890 --> 00:12:59,980 line. Correct. But then when you run them forward, you run them to like 2300. They 173 00:12:59,980 --> 00:13:02,860 all are slightly different. So they all start producing different physics and 174 00:13:02,860 --> 00:13:07,550 suddenly you get a huge like red band. So that's saying you have lots of model 175 00:13:07,550 --> 00:13:12,870 uncertainty. So it's kind of on some people might say like oh this like tuning 176 00:13:12,870 --> 00:13:17,510 process is like optimization. It's like not very scientific to be kind of right. 177 00:13:17,510 --> 00:13:22,440 It's kind of like in the past. It's kind of like the best live we've had. But I 178 00:13:22,440 --> 00:13:26,420 think, you know, we should be able to do a little bit better. Better than that. So 179 00:13:26,420 --> 00:13:28,700 just to give you the idea, you know, some people would say, you know, why don't you 180 00:13:28,700 --> 00:13:31,650 just you know, most of the physics, which you just, you know, resolve all the 181 00:13:31,650 --> 00:13:36,640 physics, you know, but see if you want to do like a direct numerical simulation, so 182 00:13:36,640 --> 00:13:39,410 it's basically saying you want to resolve all the motions in the ocean, in the 183 00:13:39,410 --> 00:13:43,310 atmosphere. You basically need to resolve things down to like one millimeter. So if 184 00:13:43,310 --> 00:13:46,850 you have like a grid spacing of one millimeter and you consider the volume of 185 00:13:46,850 --> 00:13:50,760 the ocean and the atmosphere, you basically say you need like 10 to the 28 186 00:13:50,760 --> 00:13:55,470 grid points. You know, that's like imagine putting cubes of like one millimeter 187 00:13:55,470 --> 00:13:56,850 everywhere in the ocean and atmosphere. 188 00:13:56,850 --> 00:13:58,630 That's how many great points you would need. 189 00:13:58,630 --> 00:14:01,830 So unfortunately, you could do that. But there's not enough computer power or 190 00:14:01,830 --> 00:14:05,800 storage space in the world to do that. So you're kind of stuck doing something a bit 191 00:14:05,800 --> 00:14:11,120 coarser. Usually most climate models, he's like 10 to the 8 great points so that you 192 00:14:11,120 --> 00:14:16,390 10 to the 20 to little words. You don't want to just run a big climate model once 193 00:14:16,390 --> 00:14:19,870 you know you need to run them for very long times, usually like you run them for 194 00:14:19,870 --> 00:14:23,290 a thousand years or ten thousand years when you want to run many of them because 195 00:14:23,290 --> 00:14:27,170 you want to collect statistics. So generally you don't run at the highest 196 00:14:27,170 --> 00:14:31,460 resolution possible. You run kind of at a lower resolution so you can run many, many 197 00:14:31,460 --> 00:14:36,900 models. So because you can only use so much resolution, it seems that power 198 00:14:36,900 --> 00:14:39,790 transitions or these kind of mathematical things, you have to live with them, you've 199 00:14:39,790 --> 00:14:45,310 got to use them. But at least one idea is, you know, instead of using numbers that 200 00:14:45,310 --> 00:14:49,030 sum. But he came up with in 1994. You might as well try to figure you know, 201 00:14:49,030 --> 00:14:50,980 better numbers or maybe you know if the numbers 202 00:14:50,980 --> 00:14:52,210 are kind of different in different places, 203 00:14:52,210 --> 00:14:55,910 you should find that out. So one thing you could do, one thing we are 204 00:14:55,910 --> 00:15:01,340 trying to do is get the pressurization is to kind of agree with like basic physics 205 00:15:01,340 --> 00:15:05,210 or agree with observations. So we have lots of observations. How many we can run 206 00:15:05,210 --> 00:15:07,300 kind of high resolution simulations to resolve 207 00:15:07,300 --> 00:15:09,100 a lot of the physics and then make sure 208 00:15:09,100 --> 00:15:12,100 when you put the prioritization in the climate model, it actually gives you 209 00:15:12,100 --> 00:15:16,840 the right numbers according to basic physics or observations. But sometimes 210 00:15:16,840 --> 00:15:20,810 that might mean, you know, different numbers in the Atlantic, in the Pacific or 211 00:15:20,810 --> 00:15:24,480 different numbers for the winter and the summer. And you have to run many high 212 00:15:24,480 --> 00:15:28,660 resolution simulations to get enough data to do this. But indeed, you know, these 213 00:15:28,660 --> 00:15:33,890 days I think we have enough computing power to do that. So it's kind of do all 214 00:15:33,890 --> 00:15:37,070 these high resolution simulations. We ended up building a new kind of ocean 215 00:15:37,070 --> 00:15:42,370 model that we run on GPUs because these are all faster for giving us these 216 00:15:42,370 --> 00:15:46,520 results. So we ended up usually most climate modeling is done in Fortran. We 217 00:15:46,520 --> 00:15:53,310 decided to go with with Julia for a number of reasons, which I'll talk about. But the 218 00:15:53,310 --> 00:15:58,220 left figure is kind of that mixed layer or boundary layer turbulence kind of movie. 219 00:15:58,220 --> 00:16:01,630 But instead of the rainbow color map, now it's using a more reasonable color maps. 220 00:16:01,630 --> 00:16:07,050 It looks like the ocean, the right is that old movie. So we're generating tons and 221 00:16:07,050 --> 00:16:11,670 tons of data from using simulations like this and then hopefully we can get enough 222 00:16:11,670 --> 00:16:15,440 data and like figure out a way to explain the prior transitions. But it's kind of a 223 00:16:15,440 --> 00:16:19,810 work in progress. So a different idea that might be more popular here, I don't know. 224 00:16:19,810 --> 00:16:25,770 Is instead of kind of using the existing permanent positions, you could say, OK, 225 00:16:25,770 --> 00:16:29,960 well, now you have tons and tons of data. Maybe you just throw in like a neural 226 00:16:29,960 --> 00:16:34,620 network into the differential equations. Basically, you put in the physics, you 227 00:16:34,620 --> 00:16:37,760 know, and then the neural network is responsible for the physics you don't 228 00:16:37,760 --> 00:16:43,780 know. So, for example, you know, most people here might not. I also don't want 229 00:16:43,780 --> 00:16:46,230 to talk about differential equations because I would take a long time. So just 230 00:16:46,230 --> 00:16:48,280 imagine that the equation in the middle is kind of 231 00:16:48,280 --> 00:16:49,960 what a climate model needs to solve. 232 00:16:49,960 --> 00:16:53,240 And the question marks are kind of physics we don't know. So we don't know 233 00:16:53,240 --> 00:16:58,720 what to put there. But maybe you could put out a neural network. So number one is 234 00:16:58,720 --> 00:17:03,320 kind of a possible characterisation or a possible way you could try to franchise 235 00:17:03,320 --> 00:17:05,750 the missing physics where the neural networks kind of responsible for 236 00:17:05,750 --> 00:17:09,940 everything. We find that doesn't work as well. So instead, maybe you tell it some 237 00:17:09,940 --> 00:17:13,990 of the physics, maybe tell it about cue, which is like the heating or cooling at 238 00:17:13,990 --> 00:17:17,540 the surface. And then it's kind of responsible for resolving the other stuff. 239 00:17:17,540 --> 00:17:22,300 But it's still a work in progress because the blue is kind explosive your data. The 240 00:17:22,300 --> 00:17:25,690 orange is supposed to be the narwhal and they don't agree. So it's still a work in 241 00:17:25,690 --> 00:17:28,960 progress, but hopefully we'll be able to do that better. So this is kind of stuff 242 00:17:28,960 --> 00:17:34,400 that's like a week or two old. But kind of reach a conclusion, at least from my half 243 00:17:34,400 --> 00:17:39,750 of the talk. So the reason I personally like Julia as a climate modeler is we were 244 00:17:39,750 --> 00:17:45,000 able to kind of build an ocean model from scratch basically in less than a year. And 245 00:17:45,000 --> 00:17:47,471 one of the nice things is that the user interface 246 00:17:47,471 --> 00:17:49,561 or the scripting and the model backend is all in one language, 247 00:17:49,561 --> 00:17:52,030 whereas in the past used to usually write the high 248 00:17:52,030 --> 00:17:57,850 level and like Python and maybe the back end is like Fortran or C. And we find, you 249 00:17:57,850 --> 00:18:01,400 know, when we Julia, it's just as fast as our legacy model, which was written in 250 00:18:01,400 --> 00:18:07,230 Fortran. And one of the nicest things was that basically able to write code once and 251 00:18:07,230 --> 00:18:10,510 using there's a need of GPU compiler. So basically you write your code one single 252 00:18:10,510 --> 00:18:15,180 code base and you go to CPUs and GPUs. So you'd want to write two different code 253 00:18:15,180 --> 00:18:20,610 bases. And yeah, we find generally because it's high level language, we're all kind 254 00:18:20,610 --> 00:18:24,860 of more productive. We can give a more powerful user API and Julia kind of has a 255 00:18:24,860 --> 00:18:30,240 nice multiple dispatch backend so that we find that makes it easy for the users to 256 00:18:30,240 --> 00:18:35,330 kind of extend the model or hack the model. And there's. Some people would say 257 00:18:35,330 --> 00:18:38,480 the Julia community is pretty small. But we find there's a pretty big Julia 258 00:18:38,480 --> 00:18:41,470 community interest in scientific computing. So we fund kind of all the 259 00:18:41,470 --> 00:18:46,540 packages we need are pretty much available. So with our client conclude my 260 00:18:46,540 --> 00:18:50,240 half by saying there is most of the uncertainty in climate modeling basically 261 00:18:50,240 --> 00:18:53,850 comes from humans because they don't know what humans will do. But there's a huge 262 00:18:53,850 --> 00:18:57,550 model uncertainty basically because of physics we don't understand or physics, 263 00:18:57,550 --> 00:19:01,610 the kind of model cannot see. You can't resolve every cloud and you know, every 264 00:19:01,610 --> 00:19:05,390 wave in the oceans you've got you've got to figure out a way to account for them. 265 00:19:05,390 --> 00:19:09,940 So that's what our prioritization does. And we're trying to kind of use a lot of 266 00:19:09,940 --> 00:19:14,680 computing power to kind of make sure we train or come up with good privatizations 267 00:19:14,680 --> 00:19:19,600 instead of kind of tuning the model at the end. And we're hoping this will lead to 268 00:19:19,600 --> 00:19:23,750 better climate predictions. Maybe you will. Maybe you won't. But at least, you 269 00:19:23,750 --> 00:19:28,990 know, even if it doesn't. Hopefully we can say we got rid of the model tuning problem 270 00:19:28,990 --> 00:19:33,110 and hopefully we can make. We find it that software development for climate 271 00:19:33,110 --> 00:19:37,930 modeling is easier than if we did it in Fortran. I will say this kind of an 272 00:19:37,930 --> 00:19:41,760 advertisement, but I'm looking to bike her on Germany for a week and apparently can't 273 00:19:41,760 --> 00:19:46,390 take the next bike out of Leipzig. So if anyone is looking to sell their bicycle or 274 00:19:46,390 --> 00:19:51,230 wants to make some cash, I'm looking to rent a bicycle. So yeah, if you have one, 275 00:19:51,230 --> 00:19:55,070 come talk to me, please. Thank you. Danke. 276 00:19:55,070 --> 00:20:02,190 *applause* 277 00:20:02,190 --> 00:20:09,490 Churavy: So one big question for me always is how can we ask technologists hub that 278 00:20:09,490 --> 00:20:14,580 think most of us in this room are fairly decent with computers? The internet is not 279 00:20:14,580 --> 00:20:19,410 necessarily an new island for us. But how do we use that knowledge to actually 280 00:20:19,410 --> 00:20:25,060 impact real change? And if you haven't there's some fantastic article: 281 00:20:25,060 --> 00:20:30,030 worrydreams.com/ClimateChange. Which lists all the possible or not all the possible 282 00:20:30,030 --> 00:20:36,610 but a lot of good ideas to think about and go like, okay, do my skills apply in that 283 00:20:36,610 --> 00:20:43,490 area? Well, I'm a computer scientist. I do programing language research. So how do my 284 00:20:43,490 --> 00:20:50,430 skills really apply to climate change? How can I help? And one of the things that 285 00:20:50,430 --> 00:20:54,880 took me in this article was how, and one of the realization, and why I do my work 286 00:20:54,880 --> 00:20:59,790 is that the tools that we have built for scientists and engineers, they are that 287 00:20:59,790 --> 00:21:05,550 poor. Computer scientists like myself have focused a lot on making programing easier, 288 00:21:05,550 --> 00:21:11,580 more accessible. What we don't necessarily have kept the scientific community as a 289 00:21:11,580 --> 00:21:18,250 target audience. And then you get into this position where models are written in 290 00:21:18,250 --> 00:21:23,150 a language. Fortran 74 and isn't that a nice language, but it's still not one that 291 00:21:23,150 --> 00:21:28,320 is easily picked up and where you find enthusiasm in younger students for using 292 00:21:28,320 --> 00:21:36,450 it. So I work on Julia and my goal is basically to make a scientific computing 293 00:21:36,450 --> 00:21:41,410 easier, more accessible and make it easier to access the huge computing power we have 294 00:21:41,410 --> 00:21:49,960 available to do climate modeling. Ideas, if you are interesting in this space is, 295 00:21:49,960 --> 00:21:53,590 you don't need to work on Julia necessarily, but you can think about maybe 296 00:21:53,590 --> 00:21:59,420 I'm to look at modeling for physical systems, modeling like one of the 297 00:21:59,420 --> 00:22:04,250 questions is can be model air conditioning units more precisely, get them more 298 00:22:04,250 --> 00:22:09,211 efficient? Or any other technical system. How do we get that efficiency? But we need 299 00:22:09,211 --> 00:22:15,340 better tools to do that. So the language down here as an example is modelicar. 300 00:22:15,340 --> 00:22:19,560 There is a project right now, modelicar is trying to see how we can push the 301 00:22:19,560 --> 00:22:24,770 boundary there. The language up here is Fortran. You might have seen a little bit of that 302 00:22:24,770 --> 00:22:32,310 in the talk beforehand and it's most often used to do climate science. So why 303 00:22:32,310 --> 00:22:37,100 programing languages? Why do I think that my time is best spent to actually work on 304 00:22:37,100 --> 00:22:44,250 programing languages and do that in order to help people? Well, Wittgenstein says: 305 00:22:44,250 --> 00:22:48,620 "The limits of my language are the limits of my world." What I can express is what I 306 00:22:48,620 --> 00:22:52,190 think about. And I think people are multilingual, know that that sometimes 307 00:22:52,190 --> 00:22:56,050 it's easier to think for them. But certain things in one language say it isn't the 308 00:22:56,050 --> 00:23:00,660 other one. But language is about communication. It's about communication 309 00:23:00,660 --> 00:23:05,670 with scientists, but it's also about communication with the computer. And too 310 00:23:05,670 --> 00:23:09,950 often programing language fall into that trap where it's about, oh, I want to 311 00:23:09,950 --> 00:23:15,720 express my one particular problem or I wanna express my problem very well for the 312 00:23:15,720 --> 00:23:20,420 compiler, for the computer. I won't talk to the machine. What if I found that 313 00:23:20,420 --> 00:23:24,640 programming languages are very good to talk to other scientists, to talk in a 314 00:23:24,640 --> 00:23:29,220 community and to actually collaborate? And so the project that Ali and I are both 315 00:23:29,220 --> 00:23:35,420 part of has, I think, 30 ish. I don't know. The numbers are as big as the big 316 00:23:35,420 --> 00:23:40,580 coupe of climate scientists modelers. And we have a couple of numerical scientists, 317 00:23:40,580 --> 00:23:44,650 computer scientists and engineers and we all working the same language, being able 318 00:23:44,650 --> 00:23:49,260 to collaborate and actually work on the same code instead of me working on some 319 00:23:49,260 --> 00:23:56,610 low level implementation and Ali telling me what to write. That wouldn't be really 320 00:23:56,610 --> 00:24:05,050 efficient. So, yes, my goal is to make this search easier. Do we really need yet 321 00:24:05,050 --> 00:24:09,110 another high level language? That is a question I often get. It's like why Julia? 322 00:24:09,110 --> 00:24:14,660 And not why are you not spending your time and effort doing this for Python? Well, so 323 00:24:14,660 --> 00:24:21,580 this is as a small example, this is Julia code. It looks rather readable. I find it 324 00:24:21,580 --> 00:24:28,930 doesn't use a semantic whitespace. You may like that or not. It has all the typical 325 00:24:28,930 --> 00:24:33,960 features that you would expect from a high level dynamic language. It is using the 326 00:24:33,960 --> 00:24:37,190 M.I.T. license that has a built in package manager. It's very good for interactive 327 00:24:37,190 --> 00:24:45,040 development, but it has a couple of unusual wants and those matter. You need 328 00:24:45,040 --> 00:24:49,830 if you want to simulate a climate model, you need to get top performance on a 329 00:24:49,830 --> 00:24:56,470 supercomputer. Otherwise you won't get an answer in the time that it matters. Julia 330 00:24:56,470 --> 00:25:02,830 uses just in time ahead of time compilation, the other great feature is 331 00:25:02,830 --> 00:25:07,670 actually a spitting in Julia. So I can just look at implementations. I can dive 332 00:25:07,670 --> 00:25:12,610 and dive and dive deeper into somebodies code and don't have a comprehension 333 00:25:12,610 --> 00:25:20,210 barrier. If I if you ever have spent some time and tried to figure out how Python 334 00:25:20,210 --> 00:25:26,360 sums numbers under the hood to make it reasonably fast. Good luck. It's hard. 335 00:25:26,360 --> 00:25:30,060 It's written in C. See, and there is a lot of barriers in order to understand what's 336 00:25:30,060 --> 00:25:35,380 actually going on. Then reflection and meta programing. You can do a lot of fun 337 00:25:35,380 --> 00:25:41,140 stuff which we're going to talk about. And then the big coin for me is that you have 338 00:25:41,140 --> 00:25:46,150 native keep you code generation support so you can actually take Julia code and run 339 00:25:46,150 --> 00:25:49,920 it on the GPU. You you're not relying on libraries because libraries 340 00:25:49,920 --> 00:25:59,450 only are can express the things. That was where writtenin there. So early on last 341 00:25:59,450 --> 00:26:03,550 December, I think we met up for the climate science project and after deciding 342 00:26:03,550 --> 00:26:08,450 on using Julia for the entire project. They were like, we we're happy with the 343 00:26:08,450 --> 00:26:12,710 performance, but we have a problem. We have to duplicate our code for GPUs 344 00:26:12,710 --> 00:26:20,530 and CPUs. What really? It can't be! I mean, I designed the damn thing, it should be working. 345 00:26:20,530 --> 00:26:25,559 Well, what they had at that point was basically always a copy of two functions 346 00:26:25,559 --> 00:26:31,170 where one side of it was writing the CPU code and the other side was implementing a 347 00:26:31,170 --> 00:26:36,710 GPU code. And really, there were only a couple of GPU specific parts in there. And 348 00:26:36,710 --> 00:26:43,210 if anybody has ever written GPU Code, it's this pesky which index am I calculation. 349 00:26:43,210 --> 00:26:49,710 Worthy for loop on the CPU to would just looks quite natural. And I was like, what? 350 00:26:49,710 --> 00:26:55,610 Sit. Come on. What we can do is we can just wait a kernel so he takes a body of 351 00:26:55,610 --> 00:27:00,490 the for loop, extracts it in a new function. Add a little bit of sugar and 352 00:27:00,490 --> 00:27:06,460 magic to court GPU kernels and CPU functions and then we're done. Problem 353 00:27:06,460 --> 00:27:12,730 solved. What the code roughly would look look like isn't actually this. You can 354 00:27:12,730 --> 00:27:19,670 copy and paste this and it should work. And so you have two functions. One of them 355 00:27:19,670 --> 00:27:23,679 launches where you extract your kernel. Then you write a function that takes 356 00:27:23,679 --> 00:27:29,580 another function and runs it function in a for loop or it launches that function on 357 00:27:29,580 --> 00:27:34,600 the GPU. And then you have this little GPU snippet is the only bit of us actually 358 00:27:34,600 --> 00:27:39,250 GPU, which calculates the index and then calls the function F with an index 359 00:27:39,250 --> 00:27:45,780 argument. I'm done here. My, my contribution to this project was done, 360 00:27:45,780 --> 00:27:49,650 Well, they came back to me and we're like, now it's not good enough. And I was like, 361 00:27:49,650 --> 00:27:54,550 why? Well, the issue is they needed kernel fusion. So that's the process of taking 362 00:27:54,550 --> 00:28:00,110 two functions and merging them together. I'm like, okay, fine. Why do they need 363 00:28:00,110 --> 00:28:04,790 that? Because if you want to be white(?) average efficient GPO code, you need to be 364 00:28:04,790 --> 00:28:09,980 really concerned about the numbers of global memory loads and stores. If you 365 00:28:09,980 --> 00:28:13,720 have too many of them or if they are irregular, you lose a lot of performance 366 00:28:13,720 --> 00:28:20,470 and you need good performance. Otherwise, we can't simulate the solution once. They 367 00:28:20,470 --> 00:28:24,600 also actually wanted to take use GPU functionality and low level controlled. 368 00:28:24,600 --> 00:28:29,610 They wanted to look at their kernels and use shared memory constructs. They wanted 369 00:28:29,610 --> 00:28:36,190 to do precise risk working, minimizing the number of registers used and they really cared 370 00:28:36,190 --> 00:28:40,751 about low level performance. They were like, well, we can't do this with the 371 00:28:40,751 --> 00:28:47,790 abstraction you gave us because it builds up too many barriers. And I could have 372 00:28:47,790 --> 00:28:53,970 given you a few more typical computer science answer, which would have been OK. 373 00:28:53,970 --> 00:28:59,429 Give me two years and I'll come back to you and there is a perfect solution which 374 00:28:59,429 --> 00:29:03,350 is like a cloud cover in the sky. And I write your speech spoke language that does 375 00:29:03,350 --> 00:29:06,850 exactly what you need to do. And at the end, we have a domain specific language 376 00:29:06,850 --> 00:29:10,590 for climate simulation that will do final volume and discontinuous cloaking in 377 00:29:10,590 --> 00:29:17,410 everything you want. And I will have a PhD. Kit. Fantastic. Well, we don't have 378 00:29:17,410 --> 00:29:21,880 the time. The whole climate science project that we are on has accelerated 379 00:29:21,880 --> 00:29:27,040 timeline because the philanthropist that the funding that research are. Well, if 380 00:29:27,040 --> 00:29:35,090 you can't give us better answer anytime soon, it won't matter anymore. So I sat 381 00:29:35,090 --> 00:29:40,580 down and was like, okay, I need a box. I need something. It has minimal effort. 382 00:29:40,580 --> 00:29:45,150 Quick delivery. I need to be able to fix it. If I do get it wrong the first time 383 00:29:45,150 --> 00:29:49,600 around and I did, it needs to be hackable. My collaborator needs to understand it and 384 00:29:49,600 --> 00:29:55,040 actually be able to change it. And it needs to be happened yesterday. Well, 385 00:29:55,040 --> 00:29:59,370 Julia is good at these kinds of hacks. And as I've learned, you can actually let them 386 00:29:59,370 --> 00:30:06,259 go into bespoke solutions and have better abstractions after the fact. So that 387 00:30:06,259 --> 00:30:10,220 you're that you can actually do the fancy computer science that I really wanted to 388 00:30:10,220 --> 00:30:14,860 do. The product is called GPUify Loops because I couldn't come up with a worse 389 00:30:14,860 --> 00:30:23,520 name, nobody else could. So we stick with it. It's a Macro based. And so, Julia, you 390 00:30:23,520 --> 00:30:30,730 can write syntax macros that transform the transform the written statements into 391 00:30:30,730 --> 00:30:37,330 similar statements so you can insert code or remove code if you want to. At, right 392 00:30:37,330 --> 00:30:41,330 now target CPUs and GPUs and we are talking about how do we get multi threaded 393 00:30:41,330 --> 00:30:46,480 into the story, how do we target more on different GPUs? There are other projects 394 00:30:46,480 --> 00:30:51,470 that are very similar. So there's OCCA, which is where a lot of these ideas are 395 00:30:51,470 --> 00:30:58,610 coming from and Open ACC in C++ does something really similar. But basically 396 00:30:58,610 --> 00:31:02,290 you write a for loop, you write an at loop in front of it, which is the magic 397 00:31:02,290 --> 00:31:09,480 macro that takes a transformation. And you have two indexed statements and now you 398 00:31:09,480 --> 00:31:15,090 just say I want to launch it on the GPU and it magically does a job. Get, 399 00:31:15,090 --> 00:31:22,150 fantastic. So let's pick up the entire implementation of the macro at loop 400 00:31:22,150 --> 00:31:29,270 without the error checking that didn't fit on the screen a couple of nights. So 401 00:31:29,270 --> 00:31:38,140 everything is here and basically I'm just manipulating the for loop so that on the 402 00:31:38,140 --> 00:31:46,350 GPU it only iterates one iteration per index and on CPU it iterates all of the 403 00:31:46,350 --> 00:31:50,890 indices because CPU is single threaded and a GPU is many, many 404 00:31:50,890 --> 00:31:56,810 multithreaded. Of course there's a little bit of magic hidden in the device function 405 00:31:56,810 --> 00:32:00,880 because how do I know where I'm running? And if you're curious how to do that and 406 00:32:00,880 --> 00:32:06,780 then we can talk after afterwards. But otherwise, it's a very simple, 407 00:32:06,780 --> 00:32:11,440 straightforward transformation. It's written in Julia. It's a Julia function. 408 00:32:11,440 --> 00:32:17,600 And. Yeah. So you don't need to understand the code here. I just want to show how quick it 409 00:32:17,600 --> 00:32:24,760 can be to write something like this. If you know anything about GPU Programming at 410 00:32:24,760 --> 00:32:28,620 all, there should be a little voice in the head, of the back of your head is like, 411 00:32:28,620 --> 00:32:34,780 wait a second. How can you run a dynamic programming on a GPU? That shouldn't be 412 00:32:34,780 --> 00:32:43,100 possible. Well, Julia can run on the GPU because it has a lot of meta programing 413 00:32:43,100 --> 00:32:47,770 facilities for the port for stage programing. So I can generate code based 414 00:32:47,770 --> 00:32:51,679 on a specific call signature. It has introspection, reflection mechanisms that 415 00:32:51,679 --> 00:32:56,760 allow me to do some interesting stuff in the background. It is built upon LVM, 416 00:32:56,760 --> 00:33:02,470 which is a common compiler infrastructure. And so I can actually write staged 417 00:33:02,470 --> 00:33:08,440 function that would generate an LVM specific code for my one function and do 418 00:33:08,440 --> 00:33:16,820 so do that during compile time and is a dynamic language that tries really hard to 419 00:33:16,820 --> 00:33:20,480 avoid runtime uncertainties. And this is one of the challenges if you're getting 420 00:33:20,480 --> 00:33:26,900 into Julia is to understand that when you're writing code that has a lot of 421 00:33:26,900 --> 00:33:32,230 runtime uncertainties, you get relative slow performance, or as fast as Python. 422 00:33:32,230 --> 00:33:36,780 But if you work with the compiler and you write runtime uncertainties you can get 423 00:33:36,780 --> 00:33:40,490 very fast code and you can run your code on the GPU, you basically that's the 424 00:33:40,490 --> 00:33:45,340 limites test. If you can run your code on the GPU, that you did your job well and it 425 00:33:45,340 --> 00:33:50,750 provides tools to understand the behavior of your code. So a warning runtime 426 00:33:50,750 --> 00:33:55,130 uncertainty. It does that and I don't have the time to go too deep into the answers. 427 00:33:55,130 --> 00:33:59,240 There is actually a paper about this. It has a type system that allows you to do 428 00:33:59,240 --> 00:34:03,260 some sophisticated reasoning type influence to figure out what your code is 429 00:34:03,260 --> 00:34:07,660 doing. Mutable dispatchers actually helping us quite a lot in making it easier 430 00:34:07,660 --> 00:34:11,409 to do virtualized codes. It was a case of specialization and just in time 431 00:34:11,409 --> 00:34:18,250 compilation. And so just looking a little bit closer at some of these topics, if you 432 00:34:18,250 --> 00:34:25,060 want to look at the entire pipeline that flow when you start while you're 433 00:34:25,060 --> 00:34:30,130 functioning, call it what happens through the Julia compiler. You have tools to 434 00:34:30,130 --> 00:34:33,830 introspect and all of these on the right hand side here and then you have tools to 435 00:34:33,830 --> 00:34:43,300 interact on the left hand side. You can inject code back into the compiler. The 436 00:34:43,300 --> 00:34:48,490 other thing is Julia has dynamic semantics. So when you difficult, you can 437 00:34:48,490 --> 00:34:54,200 at runtime, redefine your function and recall it new function and it uses 438 00:34:54,200 --> 00:35:00,930 multiple dispatch. So if you look at the absolute value call here, which of the 13 439 00:35:00,930 --> 00:35:06,280 possible methods will it call? In C++ or in other programing languages this called 440 00:35:06,280 --> 00:35:10,930 a virtual function call. So isn't Julia everything a virtual functional call? No. 441 00:35:10,930 --> 00:35:18,250 This is one of the important points is when we call a function, let's say because 442 00:35:18,250 --> 00:35:24,230 sign of X, we look at the type of the input arguments and then we first of all 443 00:35:24,230 --> 00:35:33,590 look at which function is applicable to our input argument. So in this case, it 444 00:35:33,590 --> 00:35:41,920 would be the real down here because float 64 is a subtype of real. So we choose the 445 00:35:41,920 --> 00:35:50,740 right method using dispatch and then we specialize that method for the signature. 446 00:35:50,740 --> 00:35:54,320 So the rule in multiople dispact is to remember is we calling the most specific 447 00:35:54,320 --> 00:35:58,670 method, whatever specific might mean. So if you have this bit of example, where we 448 00:35:58,670 --> 00:36:06,220 have a function F, which has three different methods and we have an integer 449 00:36:06,220 --> 00:36:10,960 argument that can be matched on X, or on Y, and then we have a floating point argument 450 00:36:10,960 --> 00:36:16,690 on Y and we call this with a "1,Hello". Well, we will select the methods that is 451 00:36:16,690 --> 00:36:24,420 most specific for this argument, which would be the number 1 here. On the other 452 00:36:24,420 --> 00:36:28,820 hand, if when we have a float 64 and the second position, then we will call the 453 00:36:28,820 --> 00:36:34,730 second method. Now what happens if I pass in an integer and the first position and a 454 00:36:34,730 --> 00:36:38,859 floating point in the second position? Well, you would get a run time error 455 00:36:38,859 --> 00:36:44,180 because we can't make this decision. What is the most specific method? That's just 456 00:36:44,180 --> 00:36:49,170 something to keep in mind. Method specialization works really similarly when 457 00:36:49,170 --> 00:36:55,350 you call a method for the first time. This method sign right now has no 458 00:36:55,350 --> 00:37:01,380 specializations. And then I look back, call it once and Julia will insert a 459 00:37:01,380 --> 00:37:06,710 speciallisation just for Float64. Before that it could have been a Float32. The 460 00:37:06,710 --> 00:37:11,430 Float64 is for this method. So Julia specializes in compilers methods on 461 00:37:11,430 --> 00:37:15,050 concrete called signatures instead of keeping everything dynamic or everything 462 00:37:15,050 --> 00:37:23,210 ambiguous. You can introspect this process and there are several macros that are code 463 00:37:23,210 --> 00:37:30,510 lowered or code type that will help you understand that process. I think I don't 464 00:37:30,510 --> 00:37:35,160 have enough time to go into detail here, but just as a note, if you have a look at 465 00:37:35,160 --> 00:37:40,600 this, the percentage for means it's an assignment. So if you reference it later, 466 00:37:40,600 --> 00:37:48,660 so in line 5, we will iterate on the 4 value. And then we can look at the type 467 00:37:48,660 --> 00:37:52,730 information that Julia infers out of that call. We're calling the function mandel 468 00:37:52,730 --> 00:37:59,490 with the U in 32 and you can see how that information propagates through the 469 00:37:59,490 --> 00:38:05,109 function itself. And then if you actually do agressive inlining .., we do aggressive 470 00:38:05,109 --> 00:38:09,750 inlining and optimizations and devirtualization. And so in the end, we 471 00:38:09,750 --> 00:38:15,960 don't have calls anymore. We only have the intrinsics that Julia provides on which 472 00:38:15,960 --> 00:38:23,870 programs are actually implemented. So this is a unsigned less than integer function. 473 00:38:23,870 --> 00:38:27,830 So we are using time and find as an optimization to find static or near static 474 00:38:27,830 --> 00:38:32,260 site programs. It allows us to do us agressive virtualization, inlining and 475 00:38:32,260 --> 00:38:36,630 constant propagation. But it raises problems of cash and validation. So in 476 00:38:36,630 --> 00:38:42,250 bygone days, this used to be the case. I could define a new function G after 477 00:38:42,250 --> 00:38:48,140 calling G want a function, a new function, f after calling G once and I would get the 478 00:38:48,140 --> 00:38:53,250 old restore back. That's bad. That's counter-intuitive. That's not dynamic. So 479 00:38:53,250 --> 00:38:59,560 in Julia 1.0 and I think 0.5 and 0.6 already. We fix that. So we invalidating 480 00:38:59,560 --> 00:39:06,400 the functions that have dependencies on the function. We just changed. But can we 481 00:39:06,400 --> 00:39:09,710 see latency of your program? If you change a lot of the functions and you recall them 482 00:39:09,710 --> 00:39:20,781 well hm we need to do a lot of work every time. We do constant propagation, so it 483 00:39:20,781 --> 00:39:27,420 isn't very simple example. We try to reduce. We try to exploit as much 484 00:39:27,420 --> 00:39:32,210 information as possible. And so if you call if you want a function F and you call 485 00:39:32,210 --> 00:39:36,360 the function sign with a constant value, we actually build just turning you the 486 00:39:36,360 --> 00:39:40,520 constant avoiding the calculation is the sine entirely. And that can be very 487 00:39:40,520 --> 00:39:49,930 important during hot calls and in a cycle. This can sometimes go wrong or Julia can 488 00:39:49,930 --> 00:39:53,930 has heuristics in order to decide when or whether or not these optimizations are 489 00:39:53,930 --> 00:40:00,680 valuable. And so when you introspect your code, you might see the results that are 490 00:40:00,680 --> 00:40:05,670 not that are not quite, what you want. So we don't know what the return value here 491 00:40:05,670 --> 00:40:10,150 is. It's just a tuple. We know it's a tuple, nothing else. Holistic to say, not 492 00:40:10,150 --> 00:40:14,060 specialize. But the nice thing about Julia and where we get performance voice that we 493 00:40:14,060 --> 00:40:17,970 can actually do for specialisation and hopefully at some point view makes a 494 00:40:17,970 --> 00:40:26,050 compiler smart enough that these edge cases disappear. So I can use some secrets 495 00:40:26,050 --> 00:40:33,280 and foresee specialization to happen and then I can actually infer the precise of 496 00:40:33,280 --> 00:40:40,270 return type of my function. Another thing to know when you're coming for more 497 00:40:40,270 --> 00:40:45,050 traditional object oriented programing language is that types are not extensible, 498 00:40:45,050 --> 00:40:50,190 extendable. So you can't inherit from something like Int64. You can only subtype 499 00:40:50,190 --> 00:40:55,710 abstract types. We do that because otherwise we couldn't do a lot of 500 00:40:55,710 --> 00:41:01,110 optimizations. When we, when we look at programms, we can't never assume that you 501 00:41:01,110 --> 00:41:05,310 won't add code. We had a dinamic programming language at any time in the runtime of your 502 00:41:05,310 --> 00:41:11,760 program you can't add code. And so we don't have close word semantics, which doesn't 503 00:41:11,760 --> 00:41:16,500 doesn't allow us to say, hey, by the way, we know all possible subtypes here. You 504 00:41:16,500 --> 00:41:20,800 might add a new type. Later on by saying a common types are not extendable. 505 00:41:20,800 --> 00:41:27,740 We get a lot of the performance back. So personally, for me, why do I like Julia? 506 00:41:27,740 --> 00:41:32,510 Or why do I work on Julia? It works like Pyphon, it talks like Lisp and runs like 507 00:41:32,510 --> 00:41:38,120 Fortran. That's my five sales pitch. It's very hackable and extendable. 508 00:41:38,120 --> 00:41:46,500 I can poke at the internals and I can bend them if I need to. It's a 509 00:41:46,500 --> 00:41:52,330 bit of upon LVM. So in reality, for me as a compiler writer, it's my favorite LVM 510 00:41:52,330 --> 00:41:59,190 front end. I can get the LVM code that I need to actually run. But for users, 511 00:41:59,190 --> 00:42:04,870 that's hopefully not a concern. If you do our job right and it has users in 512 00:42:04,870 --> 00:42:09,430 scientific computing and I'm in a prior life whilst doing a lot of scientific 513 00:42:09,430 --> 00:42:15,040 computing in cognitive science wanting models. And I care about these users 514 00:42:15,040 --> 00:42:21,190 because I've seen how hard it can be to actually make progress when the tools you 515 00:42:21,190 --> 00:42:28,780 have are bad. And my personal goal is to enable scientists and engineers to 516 00:42:28,780 --> 00:42:37,500 collaborate efficiently and actually make change. Julia is a big project and Climate 517 00:42:37,500 --> 00:42:44,450 is a big project and many people to thank. And with that, I would like to extend you 518 00:42:44,450 --> 00:42:50,180 an invitation if you're interested. There is juliacon every year. Where you have a 519 00:42:50,180 --> 00:42:57,830 develop meet up. Last year we were about 60 people are much smaller than CCC. But 520 00:42:57,830 --> 00:43:02,240 next year it will be in Lisbon. So come join us if you're interested and if you 521 00:43:02,240 --> 00:43:05,970 want to meet scientists who have interesting problems and are looking for 522 00:43:05,970 --> 00:43:08,570 solutions. Thank you. 523 00:43:08,570 --> 00:43:17,120 *Applaus* 524 00:43:17,120 --> 00:43:23,860 Herald A: Time for questions and answers, are there any questions? 525 00:43:23,860 --> 00:43:29,050 Herald H: Yeah, we've got microphones over there. So just jump to the microphone and 526 00:43:29,050 --> 00:43:33,010 ask your questions so that everybody could hear. 527 00:43:33,010 --> 00:43:38,510 Question: What do you mean when you say dead? Julia talks like Lisp and how is 528 00:43:38,510 --> 00:43:43,280 that a good thing *Lachen* Churavy: Well, it talks like Lisp, but it 529 00:43:43,280 --> 00:43:48,490 doesn't look like Lisp. I assume that's what you mean. It doesn't have that many 530 00:43:48,490 --> 00:43:53,960 braces. But no, Lisp has another powerful meta programming capabilities and macros. And 531 00:43:53,960 --> 00:43:58,320 so we have a lot of that. If you read a little bit about the history of Lisp. The 532 00:43:58,320 --> 00:44:03,650 original intention was to write NLisp, which would be Lisp with a nice syntax. And 533 00:44:03,650 --> 00:44:07,320 I think Julia is my personal is NLisp. It has all these nice features, but it 534 00:44:07,320 --> 00:44:13,390 doesn't have the packet syntax. Herald A: OK. Thank you. 535 00:44:13,390 --> 00:44:18,580 Question: Thanks for the talk. My question is regarding the first part of the talk. 536 00:44:18,580 --> 00:44:22,920 You, if I understand correctly, you simulating a deterministic system there. 537 00:44:22,920 --> 00:44:26,180 So there's no additional noise term or anything, right? 538 00:44:26,180 --> 00:44:29,990 Ramadhan: Well, if you had infinite precision, I think it would be 539 00:44:29,990 --> 00:44:34,850 deterministic. But I think by kind design turbulence itself is not deterministic. 540 00:44:34,850 --> 00:44:38,400 Well, it's a chaotic system, Question: But the district size version 541 00:44:38,400 --> 00:44:41,400 itself is deterministic. You don't have the monte carlo part where you have 542 00:44:41,400 --> 00:44:44,470 some noise that you would add to which might actually be justified 543 00:44:44,470 --> 00:44:49,800 from the physics side. Right? Ramadhan: Well, I mean, we, if you think if 544 00:44:49,800 --> 00:44:53,650 you ran the same simulation again, you would not get that. Well, I think if you 545 00:44:53,650 --> 00:44:55,940 ran on the exact same machine, you would get the 546 00:44:55,940 --> 00:44:58,240 same answer. So in that sense, it is deterministic. 547 00:44:58,240 --> 00:45:00,910 But if you ran on a slightly different machine like truncation 548 00:45:00,910 --> 00:45:04,010 error, I'd like the 16th decimal place could give you a completely different 549 00:45:04,010 --> 00:45:08,180 answer. Question: Sure. So the point I'm trying. Am I allowed to continue? 550 00:45:08,180 --> 00:45:12,270 Herald H: Yes, of course. There's no one else. Well, there is one person else. So you 551 00:45:12,270 --> 00:45:17,250 can continue a few minutes if you want to. Thanks. *Laughter* 552 00:45:17,250 --> 00:45:20,030 Question: So the point I was trying to make is, 553 00:45:20,030 --> 00:45:22,420 if you add noise in the sense that it's a physical 554 00:45:22,420 --> 00:45:24,790 system, you have noise in there, it might actually allow you to 555 00:45:24,790 --> 00:45:28,890 solve a PDI or discretize a PD, but get a stochastic simulation itself, which might 556 00:45:28,890 --> 00:45:34,480 be interesting because it often can make things easier. And also, you mentioned 557 00:45:34,480 --> 00:45:39,010 neural differential equations, right? And in particular, with physical systems, if 558 00:45:39,010 --> 00:45:43,231 you have an discontinuities, for example, the DT integral can actually be quite the 559 00:45:43,231 --> 00:45:47,890 problem. And there is work on to just plug my colleagues work, control neutral 560 00:45:47,890 --> 00:45:50,860 differential equations where you can actually also built in these 561 00:45:50,860 --> 00:45:53,580 discontinuities, which might also be interesting for you guys. 562 00:45:53,580 --> 00:45:56,470 Ali: That's why maybe we should talk because I don't know much about that stuff 563 00:45:56,470 --> 00:45:59,690 where we're kind of just starting up. I think that so we've been doing this maybe 564 00:45:59,690 --> 00:46:03,478 hopefully continuous, but maybe we'll hit discontinuities. I don't know. We should 565 00:46:03,478 --> 00:46:07,359 talk, though. Q: And also the math is beautiful and has no sickness. It's the 566 00:46:07,359 --> 00:46:10,359 physics that mightn't change. I'm a mathematician. I have to say that. Ali: I know 567 00:46:10,359 --> 00:46:15,480 that the physics is ugly, trust me. Churavy: Just as quickly, we do have 568 00:46:15,480 --> 00:46:24,570 stickers and I sell cookies, too. They are in the cookie box and on. I think they for 569 00:46:24,570 --> 00:46:29,530 somebody from our community is giving a juliaworkshop and we're trying to find a 570 00:46:29,530 --> 00:46:34,290 set up an assembly space and hopefully that goes out as well. 571 00:46:34,290 --> 00:46:38,490 Herald H: Go on please. Question: Also, one question for the first 572 00:46:38,490 --> 00:46:44,840 part of the talk I want. I wanted to ask if it's possible or if you are using 573 00:46:44,840 --> 00:46:49,190 dynamic resolution in your climate models. Well, you will maybe have a smaller grid 574 00:46:49,190 --> 00:46:54,900 size near the (???) and larger in the areas that are not that 575 00:46:54,900 --> 00:46:58,140 interesting. Ramadhan: Like adaptive grids? So I 576 00:46:58,140 --> 00:47:02,520 think we mostly do that in the vertical. So usually in the ocean, the thinking 577 00:47:02,520 --> 00:47:04,260 things are interesting in the, you know, 578 00:47:04,260 --> 00:47:06,070 close to the surface. We have more resolution 579 00:47:06,070 --> 00:47:08,780 there. But as you go deeper, things get less interesting. So you put 580 00:47:08,780 --> 00:47:14,380 less resolution there. Generally, I think in general, the idea people have asked 581 00:47:14,380 --> 00:47:17,760 that before, you know, why do you always use constant grids? Why don't you use 582 00:47:17,760 --> 00:47:21,700 these adaptive grids on your global, you know, models? And you the answer I've 583 00:47:21,700 --> 00:47:24,670 heard I don't know if it's very convincing. I think generally there hasn't 584 00:47:24,670 --> 00:47:28,950 been that much research or people who do research into adaptive grids for kind of 585 00:47:28,950 --> 00:47:34,960 models. Their funding gets cut. But I like the answer I've heard is a lot of the 586 00:47:34,960 --> 00:47:38,070 time, a lot of the atmosphere and ocean is turbulent. So if you especially you do 587 00:47:38,070 --> 00:47:42,750 kind of adaptive refinement, then you just kind of adapt everywhere because there's 588 00:47:42,750 --> 00:47:48,400 kind of turbulence everywhere. But yeah, I don't I'm not. I guess first for our 589 00:47:48,400 --> 00:47:53,320 simulations we're kind of just some of the numerical methods are only fast if you 590 00:47:53,320 --> 00:47:57,070 run it on a regular grid. So that's the reason we don't use adaptive 591 00:47:57,070 --> 00:48:01,310 grids for our simulations. But in general, adaptive grids for climate models is 592 00:48:01,310 --> 00:48:05,010 interesting beyond like it seems like there needs to be more research in that 593 00:48:05,010 --> 00:48:07,990 area. So I don't know if I answered your question, but I kind of just ranted it. 594 00:48:07,990 --> 00:48:10,890 Question: You did, thanks. Herald H: Go go ahead, please. 595 00:48:10,890 --> 00:48:16,160 Question: Yeah, it's just a few guesses about us. I think I have. I wept quite a 596 00:48:16,160 --> 00:48:22,170 bit of legacy fortune code in Python. And my question is, would there be a simple 597 00:48:22,170 --> 00:48:28,740 pass converting Fortran code to Julia, preferably automatically. Do you have any 598 00:48:28,740 --> 00:48:33,060 ideas about this one? Churavy: You can do it. Your Julia code 599 00:48:33,060 --> 00:48:39,100 will look like Fortran code. So you haven't won anything. So, yes. As a good 600 00:48:39,100 --> 00:48:42,170 starting point, you can do that. Absolutely. But you can also just call 601 00:48:42,170 --> 00:48:46,080 Fortran from Julia and then totally move over. I generally don't want people to 602 00:48:46,080 --> 00:48:50,060 rework their code, except if there's a good reason. Like starting from scratch 603 00:48:50,060 --> 00:48:55,470 sometimes helps. It can be a good reason. Or if you say the solutions, we don't have 604 00:48:55,470 --> 00:49:00,880 the necessary experts to to work with the old solution anymore. But generally, if 605 00:49:00,880 --> 00:49:04,700 you have Fortran code, I would just say, well, call Julia from Fortran or from 606 00:49:04,700 --> 00:49:11,040 Julia, get it up to speed and then start transitioning. Piece by piece. That makes 607 00:49:11,040 --> 00:49:16,770 sense? Herald H: So any more questions? No more 608 00:49:16,770 --> 00:49:23,530 questions. That's an early read. Ali Ramadhan, and Valentin Churavy, thank you very much. 609 00:49:23,530 --> 00:49:25,980 *Applaus* 610 00:49:25,980 --> 00:49:29,370 *36C3 postroll music* 611 00:49:29,370 --> 00:49:51,000 Subtitles created by c3subtitles.de in the year 2020. Join, and help us!