I hope everyone had (or is still having) wonderful holidays and that you are recharged and ready to do graphy things. I'm your host, Jennifer Reif, and I am joined today by Andreas Kollegger….AND we have a remarkable guest with us to talk about data and LLMs. Joining us is the innovative Jerry Liu, who is the co-founder/CEO of LlamaIndex.
Jennifer Reif: Welcome back, graph enthusiasts, to 2024 and a new year of GraphStuff.FM. I hope everyone had or is still having wonderful holidays that you are recharged and ready to do graph things this coming year. I'm your host, Jennifer Reif, and I am joined today by Andreas Kolleger, and we have a remarkable guest with us to talk about data and LLMs. Joining us is the innovative Jerry Liu, who is the co-founder-slash-CEO of LlamaIndex. Jerry has worked in machine learning and AI and has long been broadly interested in generative models, including LLMs. Welcome.
Jerry Liu: Thanks for having me.
Jennifer Reif: So, just to get us started and warm everybody up here, could you give us a brief overview of LlamaIndex and how it came about?
Jerry Liu: Yeah, that sounds great. LlamaIndex is a data framework for building LLM applications. The core mission of the company is to build the right tools to help developers connect their language models with their own private sources of data, whether that's personal data or organizational data. If you're an enterprise, you have databases, you have a lot of APIs, you have a lot of documents that you're working with. So our goal is to basically give developers all the tools they need to unlock insights from that data, and to really take that ChatGPT reasoning and generation power and apply that on top of your own data.
So that was pretty much the motivation for the project. We started a little over a year ago last November of 2022, and have been working on the project ever since and started the company around April. And so now we're one of the leading toolkits, especially around some of the core enterprise use cases for generative models, which is around stuff like retrieval, augmented generation, being able to connect to your vector database, load in your PDFs, and basically start asking questions and automating existing workflows. And so our goal is to basically make that very, very good both on the open source side as well as we're kind of building out an enterprise product right now. And the open source project has been there since the beginning, and it's basically one of the core toolkits that developer use to basically build these types of applications.
Jennifer Reif: Very cool. It's crazy how fast this industry has moved. Your project's only just over a year old and the company's only, what, several months, less than a year old. It's been a crazy year.
Jerry Liu: Definitely.
Jennifer Reif: So what kinds of features and things are you currently working on? I know you kind of hinted at it a little bit before where we started the episode here.
Jerry Liu: Yeah, so from the open source side, I think it's very interesting. I think there's still a lot of very interesting directions to explore. The goal from the open source side when we think about new features is there's always a lot of different things going on and I can kind of outline the different categories. And the goal really is to stay on top of the latest trends as well as create better tooling for both prototype to advanced LLM applications. So we are simultaneously keeping on top of what's going on in the space, and we're also innovating ourselves. We push out new techniques and then everybody kind of adopts these techniques to use in their own software. So for instance, stuff that involves just keeping up with what's going on, it includes pretty much every new LLM release. Now it's the holidays, there's not as many LLMs going on. Releases, but from everything from new OpenAI models to Google Gemini to this role, to pretty much every other open source and multimodal model coming out, there's basically a lot of just new use cases, benchmarks and... that we want to explore how they work and to see what use cases are good for.
So that's basically one source of just keeping up to date with latest advances. And then in terms of basically techniques that we try to push, there's a few categories. We focus a lot on advanced retrieval and advanced RAG techniques for better search and question answering over your data. That is something we're going to continue to innovate on, both improved performance as well as improved system level metrics, like cost, latency, dealing with production-level challenges. And then other categories in the space of things that we innovate on include how do we think about the relationship of fine tuning and what's the relationship between fine tuning and RAG, how do we think about better data representations? And that's precisely where working with Neo4j comes into play because these days, a lot of the existing RAG stack depends heavily on just vector indexing and retrieval. And so how do you move beyond that and think about more sophisticated forms of both organizing as well as retrieving your data for use with LLMs.
And then we're also thinking a little bit about as people move on beyond the simple stuff into more agentic behavior and using the outlines to automate more and more hops of their workflows, how can we create robust reliable systems there? So these are examples on the open source side. And then we haven't released this publicly yet, but we're also working on an enterprise platform. Some members of the engineering team basically are working on.
Andreas Kollegger: So just a few things.
Jerry Liu: Yeah.
Andreas Kollegger: It's pretty amazing. So just in the things that you just talked through there and is paying attention to GenAI news, there's so much going on in the space all the , and listening to you talk through all the things that are in your mind right now, things you're trying to keep track of and also work on, how do you sleep? How do you both keep up with what's going on and like, "Oh, what are you going to do about that? How do you incorporate that?" and also looking to the future, how do balance all that?
Jerry Liu: Yeah, I mean there's the actual answer, and I'm not sure how politically correct it is, but the truth is it is very hard work. I think everybody on the team has worked incredibly hard and they've done a fantastic job of just being able to keep up-to-date with everything that's going on, balancing between short-term urgent priorities with longer-term priorities. I think especially in this kind of chaotic environment, basically, I've told some members of the team this too, it's just like there's always a chance that something big is going to come out this week and we're just going to have to drop everything and focus on it. So for instance, OpenAI Dev Day, there were a lot of releases coming out, so we basically paused other features of the open source roadmap to just focus on supporting whatever the latest releases that OpenAI put out were.
And so there's always going to be unexpected things that pop up. And so it's good for everyone to be adaptable and resilient, and also... but be generalists, not get locked into specific things, just so that everyone becomes kind of this superpower engineer that can do a variety of AI full stack as well as backend work. And so I think, in the end, there's not really an easy way out. I think everybody is just kind of putting in a lot of work and hours into this. And if you talk to other members of the AI space, it's kind of similar. It's just like everybody is just always trying to discover new things, hack on co-applications. And I think the thing is, if you think about just this concept of burnout or kind of working a lot, I think it really has to be something that you're motivated in. And in the end, I think Sam Altman said this actually, it's like if there's momentum, then it doesn't feel like you're just really trying to count the hours in the week. And so given the kind of pace and velocity that the project has been growing, I think everybody does feel that energy and excitement. I think that's an important ingredient to making sure that we continue to feed that flywheel.
Andreas Kollegger: That's a fantastic observation. You're right. When you're doing something you love and you're really interested in, the time just flies, and sure, there's a lot of time that you spend, but it's not like you're exhausted by it, you're energized by it. It's actually you're kind of part of the wave and you're in the wave and it's all just a glorious mess of thought and it's great.
Jerry Liu: Yeah, exactly. I'm sure you and other members of the team are feeling that too.
Andreas Kollegger: Yeah, definitely happens as well. So thinking through the last year and with the continued pace of innovation that's happened over the last year and it hasn't slowed down at all, there's this kind of classic design thinking like, okay, the diamond is expanding right now, the possibilities are growing, and then there's some kind of conversions that happens later, that's usually done or portrayed in this two-phase step where it's like, okay, diverge, diverge, diverge, then finally converge. But I think in reality while you're diverging, some stuff is solidifying. Some of that stuff is going to carry forward. You kind of have a core that's kind of solidifying as you're innovating. There's still things on the fringes and the details, there's still things that'll change, but there's kind of a central part that's there. Do you feel like you've identified that at this point where, okay, this stuff, we know 80% of this is going to be solid, we're just doing 20%, or what are the ratios? Is there a core that you can identify?
Jerry Liu: That's a good question. I think it's kind of a complex one. I think there's nuances and some of it's probabilistic too. I can probably speak in terms of confidence percentages as opposed to this is something that's I think is absolutely going to be true. Okay, maybe there's different parts to this question. One is just do we see the overall hype, rate of advancement for AI continuing in the next year in 2024 and 2025 and onwards? And then the other part is basically what abstractions have really solidified and which parts are we confident about so that we can start building more stuff around that.
So on the first part, I think in terms of rate of advancement, I think I'm reasonably confident that the rate of advancement will continue. I think inevitably there will be troughs of disillusionment. I think we already kind of saw that in the past year when people started doing fancy agent stuff and then realized that wasn't super reliable. And then I think the hype around pure agentic things died down a little bit. But I can anticipate that picking back up.
But if you just think about the sheer scale of money that's getting poured into AI research, the competitive aspect of open source models like OpenAI, basically just everything in the foundation model layer to just this entire ecosystem, which includes us, the techniques to orchestrate these different components as well as new modalities like speech, multimodal stuff like images, video. I think there's a lot of room to make a lot of innovation. And the thing is you actually don't need to go too much beyond GPT4-level capabilities, as long as you make something like GPT4 a lot faster and cheaper and also multimodal. There's actually a lot more use cases to discover. And so I think I'm pretty confident in the next year, just the rate of innovation, of discovery of new use cases, kind of making certain things that weren't feasible in the past year more feasible in the next year. I'm pretty certain that will increase.
And then the other part here is just from the developer perspective, I still think that most people in 2023, in terms of the AI engineer persona, we're probably more on the early adopter side. It's the people that read Twitter every morning, they keep up to date with what's going on, what did Sam Altman tweet, what did Jim Fan, or just AI visionaries in the space. And they're really heads down, building different projects. But I think if you just look at the space of application developers and the promise of AI, which is that it really democratizes access to pretty much anybody can just make an API call without training a model, without too much compute, and they can just build really cool applications on top of this, there's a huge opportunity for just increased education of both simple to advanced use cases.
And I think we're just scratching the surface there. And I think as more and more developers join this ecosystem, we're going to just have the rate of advancement of best practices, techniques. And should we do evaluations? What's the best practice for different types of data, and what are the different things that we can create? That will increase too. So I think from this side of the rate of advancement, I think that will continue to increase both in terms of education and the accessibility of AI as well as just the sheer rate of new innovations coming in.
And then I think on the second piece, which is basically about whether or not in terms of just new innovations or, sorry, in terms of new use cases coming in, which abstractions would solidify as the innovations come in. I think in terms of the abstractions, it's very interesting. I think there are existing frameworks that are popping up in terms of RAG agents and other use cases. And a lot of people are building production stacks, which included LlamaIndex, or you pick like a LLM, you pick a vector database, you pick a data source, and you start composing your own applications and then you start iterating on that. I think that's starting to become relatively solidified. People are kind of building existing use cases and chatbots, I will say there's still always that 10 to 20% probability that some new model architecture comes out that'll disrupt the existing stack.
And the reason I say that is there's some interesting research going on that might tie this idea of retrieval and the model architecture more closely together. And if that's the case, then the existing way you orchestrate the interactions between a vector database and the LLM go away. And so if that's the case, you might have to swap out the existing stack. And that said, I think it's really a matter of time. Is that thing going to come out in the next year or the next two years or three years? Either way, if people have a more immediate need to get value, they can use an existing framework like LlamaIndex to unlock insights from their data. And so they can start building up or start utilizing some of the best practices that we've developed over the past year, even if that will change potentially in the next year or two.
So I think in terms of abstractions, I think there's some core components that are emerging, but I'm not going to discount the probability that this stuff might change in the next six months or a year.
Andreas Kollegger: A lot of great observations there. I was thinking about what we just were saying about the LLMs and some of the services will start picking up retrieval as part of the core functionality that they offer. And there's an aspect of that that's I guess kind of classic. Whenever you have adjacent technologies, you start to bleed into each other. So database, everybody in the database world is going to start to pick up some amount of LLM capabilities or whatever they can. Maybe they'll host models, who knows? And on the LLM side, they'll start to absorb a little bit, and what the right balance there is. I think it'll be interesting to see how that plays out. And it's probably hard to guess until people just try it and see what works best.
Jerry Liu: Right. I also think it's a very interdisciplinary field because as you said, everybody from the systems world is trying to learn AI. Everybody from AI is realizing the importance of different systems concepts, especially as you try to productionize this thing. And so just in terms of just greater awareness and know-how of building robust systems, I think there's still a lot of gaps and a lot of opportunity there.
Andreas Kollegger: Yeah, that makes sense. If I recall, some of your background was in recommendation systems. Is that right? You spent some time doing that or...
Jerry Liu: Yeah, that was my first job out of school, which was at Quora, as a machine learning engineer.
Andreas Kollegger: How much of that has influenced how you think about this space, I suppose? It seems like a lot of the sympathetic thinking is in that area.
Jerry Liu: Well, okay. I think on a very technical level, I learned what embeddings were, so that turns out to be very relevant, but I think the idea of recommendation system is actually pretty different than the existing RAG stack. And actually, I think, hot take, I think people maybe aren't thinking about it enough in terms of consumer-related applications. I think the way recommendation systems works is that the way the embeddings are generated is based on user behavior. It's not really based on the semantics on the content. It is too, these days. It's based on both. But just a lot of algorithms, it's more based on you're watching Netflix, you clicked on these movies, and so for another user that watched similar movies but maybe didn't watch this exact movie, would they likely watch this movie? That's basically how the recommendation system works. And so for a lot of consumer-facing applications, like stuff with a newsfeed, stuff with a lot of content like shopping, these algorithms basically power a lot of that.
And so I think if you try to combine that with this reasoning capability of large language models, I think it could be pretty interesting because the embedding representation isn't generated by the semantics. The way you do retrieval is based on what you think this user would likely watch or consume. And so just the way you do personalization and the way you generate these embeddings is using a slightly different algorithm. So especially as we think about chatbots these days, if we think about a personal assistant utilizing a user's previous behavior and they have awareness of what the user's actions are, I think there could be interesting ways to blend recommender system concepts into this idea of memory or personalization of building a chatbot for users. But that's basically the connection, but I haven't explored that too much yet.
Jennifer Reif: And that sounds actually very similar to bleed into knowledge graph land.
Jerry Liu: Yeah, I think so. I think there's one idea of basically kind of you construct a graph of different relationships within your behaviors and then you can somehow dynamically construct that over time. And then the other idea is just the actual embedding you generate for each kind of piece of content might dynamically change. And it isn't just based on the, again, the semantics on the content, it's actually based on your interactions and your pattern behavior. Yeah, there's a lot of interesting stuff there. I don't think people are talking about it enough. I actually haven't thought about it as much as I should have, but I think it's very interesting.
Jennifer Reif: Well, again, all of the space is relatively new. I think people are still trying to get their heads around what do we do with this. We at Neo4j have kind of explored into that knowledge graph land, but even we haven't figured it all out yet. So there's still a lot to learn, I think a lot of progress to be made yet.
Jerry Liu: Definitely.
Andreas Kollegger: Yeah, so the exploration that we've had on that side for sure is exactly as you're saying with the personalization for us, all that. Users like me ends up being a graph pattern match stuff because we know about the users already. We don't need to do an embedding for that, and it's as efficient or more efficient for us to do the graph pattern match for the... Okay, LLM, you've never seen me before, but the app knows people like me. And so we can actually then actually refine some of the answers, get a better relevancy score based on that. So users like me who have relevant content kind of refine down the set and then provide that back in for the context.
Jerry Liu: Yeah, that makes sense. So basically you're using these relationships to refine the retrieval process and these relationships are related to this idea of whether a user likes something or not.
Andreas Kollegger: Yeah, exactly.
Jerry Liu: Makes sense.
Andreas Kollegger: So through all this then, we've gotten through, okay, and you've covered a lot of ground already, which is pretty fantastic, and you started to touch a little bit on the use cases. I think the classic one, the kind of "Hello world" of ChatGPT kind of apps is like, "Okay, just talk to it," of course, but then with the RAG part, it's, "Okay, let's talk to our knowledge base." We've got a customer support knowledge base, I think is the kind of classic one. Rather than having the current generation, the pre-LLM version of chatting with customer support, now you can actually talk to an LLM that has access to the customer support material. That seems to be the easy... For any company who's got any kind of customer support, that's your first step into this world. Do you think that's right?
Jerry Liu: Yeah, I think so. I think typically for a lot of enterprises, they start off, one, I think a lot of companies are interested in RAG. To do RAG, they start off with an initial dataset that tends to be pretty small, so with a specific use case over a small amount of data, maybe like five, ten documents. And then they try it out. They use a framework like LlamaIndex, pick an LLM, pick a vector database, and then you get something and prototype working. I think there's a lot of challenges in trying to scale that to larger amounts of data. And we've talked to companies, especially the bigger ones. You have diverse sources of data. You don't actually quite know how to parse different sources of data. Sometimes you have very complex documents with a lot of embedded tables or charts or links or references to other documents, and you don't really have quite the knowledge or the time to really just parse apart every useful bit of information here and try to feed it into the LLM.
I would say one of the biggest use cases these days, again, just from the enterprise side, when we talk about RAG, I think it's kind of not super precisely defined, because RAG at a very basic level, it's just a very simple framework. You just do top key retrieval from a vector database and just put it into the context window, but there's really advanced rag RAG then there's agents. And the high-level use case that people care about, that enterprises care about, is kind of just good search and retrieval over your data. Like how do I ask a question and get back an answer? And that data could be arbitrarily complicated. It could be single-sided documents, could be multiple documents, could be multiple documents across unstructured, semi-structured, structured data. That's really what they care about. And I think that part actually still isn't solved yet. I think there's a lot of challenges and they're trying to really establish the best practices for these different challenges, and trying to help people, guide people through the process of developing the best architecture for their use case.
The kind of ideal goal for this type of system, this question answering system is that if you imagine just the spectrum of literally any type of question that you might want to ask, whether it's a simple one or advanced one or a multi-part one, you're able to get back an answer and you're able to get back good citations, sources. And so even on the search and retrieval side, I think there's a lot of challenges. I think when people think about agents, they typically think about things that can not only do search and retrieval and complex task handling, but also... So they not only read state, but they also write state. And that's probably the next step after this. I think a lot of our work has been thinking about agents with respect to search and retrieval. So stuff that doesn't modify state. I think that tends to be a little bit more appealing to people who are concerned about reliability or safety because worst you get is back the wrong answer, at least not changing the underlying data.
But once you start getting this thing to start automating the workflows, like send emails, like schedule calendar invites, look up people and perform actions, that's when you start getting into the adept style like AutoGPT you go in and actually try to do things. We'll probably see a bit more of that as these models get better, faster, and cheaper. I think, as I mentioned briefly, there was a lot of interest in this type of workflow automation type applications earlier on this year, but I think due to reliability and cost issues, mostly reliability, people started trying to really think about what are the best practices that actually constrain these models and make sure that they work well. So I can imagine that becoming a bigger and bigger portion of what enterprises care about going into next year.
Andreas Kollegger: Yeah, I could imagine on a selfish level, just like if the travel industry could just get on this, and I could book my flight out. I mentioned later in January when I'm going to fly out to California, if I could just say, "Here's my budget, here's the dates, just go figure it out. You know what I want, you know what I'd like," and it's either me having to do it or me having somebody that I trust having to do it. It seems like just so perfect for an LLM-powered solution.
Jerry Liu: Yeah, a hundred percent. And then the other thing is maybe this relates to this idea of prompt engineering. I think either way, and I think Logan for OpenAI actually said this recently, you're going to have to communicate with this AI assistant agent or this outline powered application. And the idea of prompting, I think, will never really go away, because even if the LLM is able to handle something under the hood and it's able to handle a lot of stuff under the hood, in the end, you still need to give it some task. You need to give it some instructions, and the limit, it's kind communicating with the human or a superintelligent human. And so you still need to be able to guide this assistant in the right way to achieve the goals that you set out for it. And so I think in the past year, people were really trying to figure out properties of different types of models.
Do I try to really insert the right tokens for Llama2 or Mistral to make sure that it's able to achieve this task well? I think as these models get better and as these cases of having this personalized assistant that can really do anything, as that vision really takes off, we're going to start to see better practices of how people are able to converse with these assistants to achieve the goals that they want. And I think that's going to be a pretty interesting thing to explore, because it's kind of at the intersection of... It's not really code at that point. It's like how you communicate and how you interact with an assistant.
Jennifer Reif: To be fair, when computers first came out, there was a learning curve to figure out, "Okay, this is basically a glorified calculator." To some degree, you have to think like a computer in order to successfully interact with it and get what you need out of it. And that to some degree has always been the case. You look at code, you again have to think in the computer's terms in order to get what you need out of it. And so there probably will be some of that just more natural language now than what we've dealt with anytime in the past. But yeah, I think you still need to somewhat understand where the computer's coming from in order to help it get you there.
Jerry Liu: Right. I guess what I'm saying is I think it's starting to converge on just human levels of communication, or maybe some level of communication that's neither human nor pure code. And then people are going to have to learn these skills of how do you actually properly use natural language to converse with AI.
Jennifer Reif: Yeah.
Andreas Kollegger: Oh, I see. But so, with some of the prompt engineering where it feels like you're kind of trying to trick the LLM to do something, that part of it will go away because the LLM will have a better sense of what you mean rather than you trying to figure out how to really tell it. No, seriously, don't do the thing, don't do this, please don't do this. And you say it 12 times to make sure it really doesn't do that.
Jerry Liu: Right, exactly. It's like there's the hacks that people are discovering right now, and then there's just this longer-term skillset of how do I actually just talk to this thing to actually... It's like having a kid or having a sibling or having a friend or having a pet that's like, "How do I guide this thing to work with me?" Or I guess it's different than having a friend because you're actually just trying to guide this thing to do what you'll tell it to do, but how do I properly converse with it so that it can help me achieve my goals?
Andreas Kollegger: Right. Really cool stuff. So, Jen, I was... just to have a quick time check here, we're into half hour. I'd love to keep this conversation going for a while, but how do you feel about our time so far?
Jennifer Reif: Yeah, we can go ahead and maybe segue into our tools of the month.
Andreas Kollegger: Cool.
Jennifer Reif: So anybody want to go first?
Andreas Kollegger: I've seen the show notes that I've got my initials up first, so I'll go. And I've been, in the last month, rediscovering frontend frameworks despite wanting to spend my time on just LLMs and GenAI. I'm like, "Oh, I still have to write frontend code." And there's a firm called Remix, which I've actually become quite fond of. Next.js and React are where people usually go for things these days or Svelte if you're being super cool, but Remix 6, really interesting approach to how it puts together front ends. It's a very nice client-server kind of coalition that kind of comes together, and it lets you kind have things that are much more... it feels much more data-driven, even though we have headless CMSs and other frameworks, we're building front ends today where you separate out the API, you define your models and things and that's great.
This feels much more like you are just defining your UI where you want data to appear, and then you separately have some loaders for like, "Okay, I don't care where that data comes from. Something else is going to take care of this at some point." And then of course you can plug into GraphQL or whatever you want to eventually, but it's kind of a, that all gets handled magically on the server side, and then just on the client side, it's just, "Okay, here's just HTML pages." You end up basically serving up HTML pages that have been hydrated with all the data that they need. There's some nice magic that happens behind the scenes. You can keep things up to date when data gets invalidated, but it's a really nice framework for just building data-driven websites, I would say. Somewhere in between how React feels like you're building applications and Next.jus is more like you're building websites. This is kind of in-between world where you're like, "It's not quite an application. It's like a data-driven website."
Jennifer Reif: Is it kind of like a Retool sort of thing?
Andreas Kollegger: I haven't spent a lot of time with Retool, so I don't know that I can say that. I thought Retool had more of a UI approach to things.
Jennifer Reif: Yeah.
Jerry Liu: Like low code.
Andreas Kollegger: Yeah.
Jennifer Reif: Okay. So this is more code probably than that?
Andreas Kollegger: This would be like Next.js if Next.js was more focused on data than on just content.
Jennifer Reif: Okay.
Jerry Liu: It's cool.
Andreas Kollegger: Not quite fair to say, but it's something like that.
Jennifer Reif: Okay, cool. For mine, kind of going back, Andreas talked last episode about a terminal that was just fabulous and raved about, so if you haven't seen that episode, go back and check it out. But it got me kind of thinking, I was playing around with some API building stuff, applications and things, and I was just using plain old cURL and realized how boring and drab cURL is in some respects. And so I went back. I have used HTTPie in the past, and that's kind of my go-to, but I was trying to find something that would be applicable and everybody would have installed. So that's why I was using cURL. I'm like, "You know, there's really some nice things about HTTPie that I just really miss." So, first of all, the data results that get back are auto formatted, auto prettyfied, which is super. The requests are very... they feel a little bit more user-friendly, I guess. Because you type HTTPie, and then if you're using localhost, you don't even have to type localhost. You just put the port number, and then your API path at the end of it.
So there's a few little things that it just simplifies and it just makes boring old command requests a little bit easier and a little bit cleaner, I guess, is what I would say. So that's kind of one of the basic things, but again, brings me joy on a daily basis doing the same old commands over and over again. So, HTTPie.
Andreas Kollegger: Cool.
Jerry Liu: Nice. I'll just talk about two things. One is more related to actual work and the second really has nothing to do with it. The first one is I've been pretty interested in just headless browsers that you can just import as a Python package. And so I discovered Pyppeteer, which turns out to be unmaintained. Now I think they tell you to use Python Playwright, but basically, why do I care about this? It's because it basically just gives vision capabilities to a multimodal model to browse websites. So if you have a headless browser, you can go up to any website, screenshot it, and feed it to [inaudible 00:33:33] Gemini Pro, and you can go in and actually just have it not just comprehend the website through parsing the HTML, but actually just parse the screenshot. And it turns out it's actually, if you feed an Amazon page or feed it a shopping page on Macy's or something, it'll be able to actually understand that from just the layout of the screenshot versus actually having to dig into the URL or, sorry, not... parse the HTML.
So I think that's been pretty interesting to explore. I think on a pure non-work related or non-AI related thing, I like efficiency when using my computer. And so I really like rectangle, which is on Mac OSX. You just use a keyboard shortcut, and it's splits your screen into a bunch of different rectangles. And then you can go in and... You can have two Chrome tabs side by side. I use it all the time when I'm on a Zoom meeting. And speaking of Zoom meetings, I've been using Fireflies.ai for just transcription and notes. And so that's been pretty nice because then you can just go in and recap things without having to take notes afterwards. And then the last thing I'll say is I really like video speed controller, which is a Chrome extension. I have never watched videos on 1X anymore. So basically for anything, any video on the internet, you just dial it up to 2X, then you can basically just speed up the rate at which you consume content.
Jennifer Reif: So I didn't used to speed up video, and Mark Needham and Michael Hunger have got me doing it, and now I can't go back. I can't listen to anything on normal speed. I always have to speed it up at least a little, like one and a quarter, one and a half at a minimum.
Jerry Liu: Exactly. Yeah.
Jennifer Reif: That would be super helpful. I may have to look into that browser extension now. Because that's always the irritating thing. You can speed it up on YouTube, And now I've noticed on some of the video embeds, you can speed them up, but if it's like a Vimeo or some of the other links, you can't always speed up the video.
Jerry Liu: Yeah. This will let you do anything.
Jennifer Reif: Ah, that's awesome.
Andreas Kollegger: Nice. Life hacks. This is how you keep up-to-date with everything. You've got podcasts at double speed, videos at double speed.
Jennifer Reif: Efficiency.
Andreas Kollegger: 48 hours a day.
Jerry Liu: Exactly.
Andreas Kollegger: Awesome.
Jennifer Reif: Okay. So, Jerry, this is where we can let you drop off if you would like, before we dive into all of our graph news. But thank you so much for joining us. I really appreciate all of your insight, and I think this was a fantastic episode, and to hear your perspective going into 2024 to talk about LLMs and kind of where everything is going, as well as LlamaIndex. So thank you very much.
Jerry Liu: Of course. Thanks for having me. Take care.
Andreas Kollegger: Great. See you, Jerry.
Jennifer Reif: Happy New Year.
Andreas Kollegger: Happy new year.
Jennifer Reif: Okay. So I didn't have anything noted in community projects for this month, but we will hopefully have something headed your way from next month. As far as product updates, there's several things going on here in the last month, but most of them all center around the fact that there was a new Neo4j release of 5.15. And so, there's a few things that are involved with that. There's actually a command now for node vector indexes. There's some changes to CDC, change data capture, and then some large transaction work being done there as well. And then of course you have things that go along with the Neo4j release, like driver updates, APAC core updates, GraphQL releases, Helm chart updates, and then several Neo4j connectors were also updated. So, again, I'll have all of those links in the show notes, but most of all of that centered around the new Neo4j 5.15 release. Anything else, Andreas, that you can think of?
Andreas Kollegger: No. I have to be honest, I'm still on the holiday mindset, so I'm trying to remember, what was happening at work? Yeah.
Jennifer Reif: Yeah, December, the last, or the first, I guess, two, three weeks of the month.
Andreas Kollegger: Yes.
Jennifer Reif: Yeah, turning the dial back over to the work setting this week. Okay. So just looking at some articles, which, again, I have not had time to go through everything yet, because again, we're on holiday time here, but just kind of doing some overview of some things. There is a new next-generation graph-native storage format in Neo4j. And so there's an article talking a little bit about that. I know internally we've seen kind of some things pop up, but this is now being talked about on an external level. So this is a blog post on the Neo4j developer blog talking about this. So feel free to check that out. If you're curious about database internals and how the engine works and all that good stuff, there should be some interesting things there.
And then going back to what we were talking about with Jerry, there's some implementing advanced retrieval RAG strategies with Neo4j. So there's an article out there for that. There's some knowledge graph stuff, some more RAG, a Neo4j and Lang chain article as well. So those are the things that have all hit the developer blog this month. I did see there's been a couple of videos added to the Nodes 2023 playlist. So if you were a part of nodes, didn't get to see some of the content or missed some things because you were in one track or a time zone or something, there's new videos popping up all of the time, so be sure to keep tabs on that. That'll be probably a recurring thing for the next month or two, I would think at least a trickle of release there. And then the other video is RAG with a Neo4j knowledge graph, how it works and how to set it up. And I actually watched this video, it's not a very super long video at all, I want to say less than 10 minutes.
And so interesting, he walks through how to get things started, what all that entails and so on there. So good video, interesting and relatively short, as you maybe are kind of closing out holidays and want to keep tabs on work-related things, or maybe when you get back you want to kind of ease that OnRamp into work things. That would be a good starter thing to have. As far as events, we've got a few things rolling in January, but nothing too major. Again, we're starting the year. After the holidays, everybody's still in their holiday comas. So January 4th, we still have the Going Meta YouTube series that will be online, virtual, live streamed. So check that out. There's on January 10th, there's a couple of things. Virtual, and then there's a meetup in Austin. It's also virtual as well. January 11th, couple of webinars happening there, talking about trends, what data analytics leaders need to know.
And then January 17th, there's an O'Reilly Media, Generative AI for Healthcare. That's also virtual, so check that out. January 22nd, yet another virtual webinar talking about chatbots, GenAI. And then January 23rd, some more virtual things. So not a lot of travel going on, so you don't have to leave your comfy warm home in January. January 25th, another YouTube series for Neo4j Live. This is something that our colleague Alex runs, and it looks like they're talking semantics-based recommender systems for ESG documents. So check that out. It's a YouTube series. And then there is a conference in Bristol, UK, Graph Talk Government, at the end of January. That's January 25th. And then closing out the month, there is a meetup on LLM knowledge graph in London, and then another meetup, which I could not find a location on that one, but it's talking cloud-native geospatial analytics, combining spatial SQL and graph data science. So those are the events that we'll close out our January. So a lot of virtual things. Be sure to check those out, and hope everyone has a nice quiet and productive January.
Andreas Kollegger: And I will give a plug for the London community. We've committed to next year, we're going to be running meetups every month out there. A lot of it's going to be GenAI focus. We're working with some GenAI groups both in London, also up in Cambridge, and sort of building the overall community in the UK which is going to be great. So if you happen to be in the UK, you're interested in graphs or GenAI, reach out to us. We'd love to have people who have great ideas. We want to talk about whether you're on the enterprise side, if you're a developer. We actually have some academics that we're reaching out to, some of the schools we're collaborating with that are going to come out to the meetups as well. So it'll be a lovely convergence in the community of research, practical applications with developers, and then of course, business value from the enterprise perspective on stuff. So if you have any thoughts on any of that, please reach out to us. We'd love to talk to you.
Jennifer Reif: That sounds like a great collaboration.
Andreas Kollegger: Should be good. I have high hopes.
Jennifer Reif: That'll be fun. Good way to start off 2024.
Andreas Kollegger: Totally. Yeah.
Jennifer Reif: All right. Well, I guess that closes us out for this episode of GraphStuff.FM, and we will talk to you all in the next month.
Andreas Kollegger: Good to see you, Jen.
Jennifer Reif: Cheers.
Andreas Kollegger: Take care. Bye.