Starburst’s Post

I've been doing this logging thing for awhile now and some things that are completely blowing me away about Observe, Inc. in my first 6mo here. 1. The rate of engineering releases and innovation is staggering. We internally collectively thought a filter on a certain screen was a good idea. 2 weeks later it was in the product and worked perfectly. Every week this product gets better and better. 2. Pay for what you use: In literally everything else i've ever used, if you ingested it, there was licence attached to it in some fashion. Observe you only pay for what you use. So if you'd like to centralize all of your data in one location but have a compliance use case where you simply need to retain certain data but don't plan on using it much, you only pay for the compressed storage of that data, which on average is around 10x and at the going S3 storage rates are right around $23/TB per month. So at 10x you figure 10TB per month would store at roughly 1TB. Amazing and its all available to query at any time if you need to through the UI. No storage tiers. 3.Every customer gets a Data Engineer. If you need a dashboard, help shaping some data, assistance with creating an alert. Just reach out! It's completely included! Wild! 4. If using one of our Apps doesn't happen to contain the utilities you need to get your data in such as our AWS app that will scrape quite literally about everything from your AWS account, shape, it and give you immediate insights or if there isn't any clear route on a collector that will get in what you need, Give us the endpoint to hit and an api key and we'll set up a collector for you in our environment! 5. I can't even begin to describe the power of GraphLink. Completely disparate data sources that might have one thing in common such as some CommunityID, Hostname, IP, Tag, they can be linked together and have fields from each be combined together to create one view without writing a single join statement. 6. #5 leads into #6 here. Intuitive. On day 1 I was building my own datasets, creating dashboards and alerts for my environment and if I happen to run into anything I wasn't sure about. Asking o11yGPT in the ui would either navigate me to where i needed to go or even write the query I was trying to accomplish for me. If interested in checking any of this out, nothing to lose. Give our free trial a go! #logmanagement #logging

DSA 15-Day Challenge: Day 1  Highlights Greetings, everyone! Today kicks off my 15-day journey into Data Structures and Algorithms (DSA), and I'm thrilled to share the highlights from Day 1 with you. Understanding Data Structures Imagine data structures as filing cabinets for computer programs – they organize and efficiently manage data for easy access. Types of Data Structures Covered: *Stacks: Similar to a stack of plates where you can only add or remove from the top (Last In, First Out - LIFO). Perfect for tasks like reversing text or managing function calls. *Queues: Think of a coffee shop line where people join at the back (enqueue) and are served from the front (dequeue) (First In, First Out - FIFO). Ideal for managing tasks in sequence. *Linked Lists: Unlike arrays, linked lists are flexible chains where each element (node) contains data and a pointer to the next element. Great for dynamic memory allocation. Understanding Efficiency: Time and Space Complexity These metrics gauge how well an algorithm performs: *Time Complexity: Measures how many steps an algorithm takes as data size increases. Like finding a book in a messy room versus an organized one. *Space Complexity: Measures extra memory an algorithm needs. Imagine the extra boxes required to organize a growing book collection. Search Algorithms: Navigating Data We explored two search techniques: *Linear Search: Checks each list element sequentially, akin to scanning a bookshelf. Suitable for smaller datasets. *Binary Search: Efficient for sorted lists by halving the search area until finding the target. Much faster for larger datasets. Day 1 offered a glimpse into the expansive world of DSA, and I'm excited for what Day 2 holds! Stay tuned for more updates on my journey, and share your DSA learning experiences in the comments below. Let's learn and grow together! #DSA #DataStructures #Algorithms #15DaysChallenge #CodingChallenge #LearningJourney #TechSkills #dsa

It's 'National get to know your customer day' So, we want to hear from you. It can be a story about your neighbours cat, or a random 'tech' question you would like to know but can't be bothered to Google. We are here for it. Ps. Did you know that on our website we have provided a learning centre where you’ll find an ever-growing collection of resources such as: Answers to common questions we get asked Answers to questions you might not have thought of, and Answers to questions that you’re too afraid to ask! https://lnkd.in/eaVvec4Z #Nationalgettoknowyourcustomerday

https://lnkd.in/eykZn8TH And so it begins! My new blog site is up. Still figuring out how to use it so its going to be a long learning curve :) Go to "about" for my first blog post. https://lnkd.in/eGGBiSjW

Explore the short course on incorporating graph-based data structures in your LLM application, presented by Wey Gu from NebulaGraph Database in collaboration with LlamaIndex. Unleash the power of #NebulaGraph to unlock new possibilities for your LLM application!

Dear Reader, I wish you a joyful Thanksgiving Day! 🎊 There's just one day left before your holidays, so let me take a moment and introduce you to one of the most actual features, like the turkey at your family dinner - Lanes Analytics. While you may currently collect your learning data in Excel or other disconnected platforms, and perhaps find a lack of resources in your existing learning software, Lanes offers something truly special - an interactive and feature-rich analytic section. 🤓 At every step of the learning process, all your data is gathered in one convenient place. Whether it's a video session, webinar, or your talented individuals expressing their ideas and voting on them, Lanes can handle it all. (No wonder our existing users adore Lanes!) And here's the cherry on top 🍒 - with just one click, you can download a custom report with variable data and easily share it with your L&D team for further analysis and improvement. Isn't that cool? I bet you're itching to explore it! You can watch a video about Lanes Analytics here: https://lnkd.in/dhPxMd9Z or simply schedule a chat with us at https://lnkd.in/dsjU8gyv

We've partnered with Wey Gu to create the world’s most comprehensive short course on using LLMs with Knowledge Graphs.  ✅ Key query techniques: text2cypher, graph RAG ✅ Automated KG construction ✅ vector db RAG vs. KG RAG All this content is contained in a single Colab notebook: https://lnkd.in/gJ3D5Kr6 There's also a full 1.5 hour video tutorial: https://lnkd.in/gDtzGwx5 It's a must watch if you're exploring how to use graph-based data structures in your LLM application!

KG are very important to various use cases, as they can contain complex relationships that other DB types cannot. This is why it is important to try and integrate LLM into the KG query loop. LlamaIndex released a quick webinar demonstrating how to just that ⬇️

I see a lot of excitement about new open LLMs being released every day: Mixtral, Mamba, StripedHyena, Zephyr, phi-2 just within last week 🔥 But just having a good LLM is not what most consumers care. LLM is an engine and consumers need a car. Here are 12 things that LLM providers should start building for their services: 1. Hallucination detection. Any made up answer has a huge cost and ruins the trust with a consumer. It’s not just improving LLM accuracy, but also detecting it post-factum and correcting it in real-time. 2. Safety guardrails. Discrimination, toxicity, biases, even using incorrect brand names or inappropriate greetings prevent businesses from using LLMs. 3. Real-time updates. LLM should have access to the up-to-date information, especially true for businesses where the catalog of products or offerings is changing on a daily basis. 4. Tool calling. No AGI can be achieved with autoregressive model itself. LLMs need to be able to operate natively Browsing, Calculators, Programming capabilities and provide access to custom function calling. 5. Retrieval support. Each and everyone have thousands of documents. It should be easy to connect these data sources with LLMs. At least add PDF upload. 6. Infinite context length. Sufficiently complex tasks performed by humans such as writing code or making a presentation requires parsing millions of tokens. It’s a no-brainer that LLMs should be able to do this too. 7. Memory. Users don’t want to repeat themselves and it’s rarely the case we get the answer with one question. LLMs should be smart enough to understand what to put in long memory and what to put in fast one. 8. User preferences. When users translate from French to English all the time, LLM should remember the choice and use it by default. Similarly, it should learn other preferences and align its responses based on them. 9. Fine-tuning. This should be no-code capability to fine-tune or align the models, because there are just so many scenarios we deal in our lives. 10. Planning. Many daily tasks such as booking a trip or buying groceries online require planning and while planning can be achieved natively by chain-of-thought it’s limited and external systems that could plan and verify the execution of LLM are needed. 11. Prompt library. At the very least, for each LLM there should be an easy access to prompts that work. More advanced providers would run LLM with different prompts automatically chosen for each request. 12. Evaluation. As we customize the models with points above, there should be an easy way to track if the performance degraded on down-stream tasks or public datasets. The progress with open-source LLMs is great and should continue, but it’s just the tip of the iceberg. It requires a lot of engineering and science work on top of LLMs that could make them reliable and useful tools.

I've returned to using DTW for time series comparison after a little break. Surprisingly, you can segment a bunch of time series with just a few lines of code and scikit-learn's Birch method. It's amazing to see how powerful combining DTW with clustering algorithms can be. I created a very simple example of how to cluster companies based on their weekly returns using DTW and Birch. The nice thing is that it runs in a few seconds and requires minimal resources. #datascience #machinelearning #timeseries https://lnkd.in/dTBw8Ans