DuckDB’s Post

Probability and Trading goes hand in hand, but which model best fit certain situations can be tricky to figure out sometimes. Black-Scholes options pricing model is well known and widely used in European markets but the model can certainly be improved. In my python work below, I have used T-distribution and Gaussian KDE modelling to fit trading losses distribution's peak and tails respectively (using data from 2008 financial crisis). Any feedback would be helpful. GARCH models can also help to separate/identify true inference in data from otherwise sampling bias. https://lnkd.in/gdVMbj-H

Hello LinkedIn, just finished the 𝐀𝐭𝐜𝐨𝐝𝐞𝐫 𝐁𝐞𝐠𝐢𝐧𝐧𝐞𝐫 𝐂𝐨𝐧𝐭𝐞𝐬𝐭 365 and was able to solve the first 5 problems and secured a rank #2429. Here's a quick explanation on all of my solution : Problem A, B were just trivial so i will explain only C, D, E : 𝐂. 𝐓𝐫𝐚𝐧𝐬𝐩𝐨𝐫𝐭𝐚𝐭𝐢𝐨𝐧 𝐄𝐱𝐩𝐞𝐧𝐬𝐞𝐬 : It was a basic binary search problem , the subsidy is monotonous i.e if some subsidy x is valid then all subsidies <= x are also valid so. Using this we can easily find the last x such it is <= M. May be we can solve this without using binary search but during contest i found binary search easier : ). 𝐃. 𝐀𝐭𝐂𝐨𝐝𝐞𝐫 𝐉𝐚𝐧𝐤𝐞𝐧 3 : I used dp to solve this problem, for this problem also there may exist a greedy solution, if anyone solved greedily do let me know in comments : ). So we will create a 2-d Dp where Dp(i, 0) if we draw at i'th index, Dp(i, 1) if we win at i'th index. So the transition is for Dp(i, 0) = if at index i - 1 we drawn the game and we wan't to draw at this index also, then s[i] != s[i - 1], if we won at index i - 1 then win(s[i - 1) != s[i] { win[c] = character needed to win against c}, we have to take the max of both of the above possibilities by checking the conditions i've mentioned. Similarly u can use this idea for defining the transition for Dp(i, 1). 𝐄. 𝐗𝐨𝐫 𝐒𝐢𝐠𝐦𝐚 𝐏𝐫𝐨𝐛𝐥𝐞𝐦 : To solve this we will add the contribution of each bit to our answer separately. We will be maintaining three 2-d arrays : 1. Start's : start(i, j) is count of such indice k < i where j'th bit was not set 2. Was set : was_set(i, 0) is is the last index where i'th bit was set, was_set(i, 1) second last index where i'th bit was set. 3. next set : next(i, j) is the index k >= i where the j'th bit is set. So now will iterate from i = 0, n : at each index we will iterate from j = 0, 29 now if j'th bit is set in a[i] then we add the contribution of all such start's where j'th bit was unset which is St = (i - was_set(j,0)) so St is the number of start's so we add St * (1 << j), then we add the contribution of all ends next to i'th index which is En = next(i, j) - i - 1, so we add En * (1 << j). Then we add St * En * (1 << j) , cause we can pair up all starts with ends. Now we check if was_set(j, 1) == -1 or not if it's not then we count the number of starts at index(was_set(j, 1)) let's denote it as St2 so we add St2 * (1 << j). And we can pair up these start's also with our ends, so for that also we add St2 * En * (1 << j). After all this we add the start's at index(was_set(j, 1) to Start(i, j) if was_set(j, 1) != -1. After doing all the stuff above we will update our was_set array accordingly. Hopefully you guys must have got the idea of the approaches i used. If you got any doubt or you don't get the solution idea clearly do let me know in comment's. #Atcoder #atcoder #codeforces #leetcode #codechef #connections #competitivecoding #beginner #cfbr #cp #coding #algorithms #algorithm

🚀 Excited to share my latest Python notebook on Analyzing Product Quality in Manufacturing! 📊🔍 In this notebook, I dive into understanding how various factors like supplier type and material quantity affect the final product quality, using data from Spectrum Shades LLC. Here are some of the key insights: 1. Identified key factors contributing to inconsistent color quality. 2. Calculated mean and standard deviation for critical variables. 3. Analyzed the correlation between pigment quantity and product quality score. To achieve this, I utilized powerful libraries such as pandas, numpy, seaborn, and scikit-learn. The notebook covers data cleaning, exploratory data analysis (EDA), and advanced statistical techniques to uncover meaningful patterns and relationships. Check out the live publication below. https://lnkd.in/g67RW3nx I would love to hear your thoughts and feedback! Feel free to connect and share your insights. Let's keep the data science conversation going! 💬📈 #DataScience #Python #MachineLearning #DataAnalysis #BigData #Analytics #DataVisualization

" ❌ LLMs are too compute-heavy- I can't deploy them with my limited budget. Even the so-called open-source models don't release their training data. So, they are like a black box to me, I can't predict their behaviors, biases, etc.❌". Not anymore! 🚀Proudly presenting MobilLlama: A Lightweight and Accurate Alternative to Gigantic Blackbox LLMs 🚀 Key Highlights: 🔍 Fully Transparent: Opensource the full training data, intermediate checkpoints, and final models. 📱 Mobile-friendly: Yes, you can run it on mobile devices without an internet connection!. 🔧 Customizable: Modify and redistribute, Apache-2.0 license 🎯 Accurate: The world's most accurate model below 1B. 🎨 Multimodal: The VLM variant can take image and text inputs! Additional Information: 📖 Dive into our paper: https://lnkd.in/g-kbuFZW 🤗 HuggingFace: https://lnkd.in/gajWQmDn 🔗 Github page and Android application: https://lnkd.in/gN3R22hX 🙏 Thanks to the entire team! Omkar Thawakar, Ashmal Vayani, Salman Khan, Muhammad Anwer Rao, Michael Felsberg, Tim Baldwin, Eric Xing, Fahad Khan

check this

If you are not following Sandeep Pawar 's blog on Fabric yet, make it a 2024 resolution to do so. DataOps emphasizes improving cycle times in delivering features, and you can lessen the learning curve to deliver by reading his great content on Fabric. In his latest article he covers cancelling dataflows gen2 referencing the various ways to do so from the community (e.g., Gerhard Brueckl and Alex Powers) https://lnkd.in/eip8CFDH

Bypassing hx-disable (yet another one from Mathias' big bag of HTMX bugs). hx-disable is SUPPOSED to disable any HTMX functionality within the children of the tag. However, the syntax of HTMX triggers has changed: OLD: hx-on=error:alert(1) NEW: hx-on:error=alert(1) This new way of specifying triggers bypasses hx-disable, for example: <div hx-disable><img src=x hx-on:error="alert(4)"></div> Even when using DOMPurify this can be tricky to defend against. With HTMX you can optionally prefix all HTMX attributes with "data-" so they look like data attributes. Everything starting with "data-" is allowed in DOMPurify!

EDIT AND CORRECTION: 😒😤😡 Turns out that Polars made a bad design choice in not explicitly mentioning that reading from a Feather file is lazy and hence, it doesn't load things into memory. Still, according to the source below, it's very useful to read from a Feather file and force it to read things into memory. SOURCE: https://lnkd.in/e8aVvmsr ========================================== Genuinely.... mind blown! 🤯 TL;DR: Use Feather V2 data caching with Polars to achieve unbelievable read/write performance. ========================================== I have datasets that are millions of rows by thousands of columns (~6.3 BILLION values). Started by using Pandas (yes, I know it's not built for this). It took 1.5 hours to load the data before needing more than 300GB of RAM. Hence, I switched to polars (https://pola.rs/). Huge blessing but still, it takes 5-6 minutes to load the data. I looked into the best data caching file formats (lots of deliberating between Parquet and Feather). Ultimately, I chose to use uncompressed Feather and the speed gains were nuts. I'm now able to load 6.3 billion values in under 10 seconds. 🫨🤯 How to read and write to feather in polars: https://lnkd.in/ecwdfH-5 https://lnkd.in/e38BpX8U

Do you want to figure out why your #rstats code is running slowly? Using a code profiler is very helpful to identify the bottlenecks in your computations. I recently came across {proffer} (https://lnkd.in/eDUKzXp9) by Will Landau and it's very easy to use and helps in understanding where most of the computation takes place. Under the hood, it uses Pprof an open source Go tool developed by engineers at Google.