Basic Pipeline

Before I could start working on this project, however, I had to seriously think about what KIND of data I needed. Since I was a frequent user of the subreddit myself, I simply thought about what I would want to see in the site, and what kind of data would be necessary.

My ideal tool would be a searchable repository of restaurants with a summary of the different reviews, as well as a rating, so I could tell at a quick glance which restaurants might interest me. Filters by location, cuisine, and potentially other data points would be helpful. Lastly, I would need a way to search by specific dishes, and where to get them. So for example: I'm in the mood for a philly cheesesteak - which restaurants in New York City have the best cheesesteak?

With this in mind, I came up with a quick list of criteria I needed. I would analyze both posts and comments and find the following:

• What restaurant they were talking about
• HOW they rated the restaurant (was it bad? terrible? OK? Amazing? The best they'd ever had?)
• What dishes did the comment/post talk about (e.g. lobster bisque, hamburger, caesar salad...), and how did they rate those cuisines?

Once I had this data, the application would be more or less a standard CRUD app.

There were a few more considerations I had to make, though, such as the linking of restaurants. I had to make sure that different variations of spelling/wording for the same restaurant were all represented by the same entity in the database (Katz's, Katz's deli, Katz's Delicatessen, etc.).

I had a similar issue for dish mentions. If a bunch of people are talking about a restaurant, and many said it has excellent burgers, how do I make sure all of these burgers refer to "burger" in the database, so this data can be used to score restaurants accurately and users can see which restaurants have the best burgers?

The last thing I would have to do would be to rate the restaurants. I decided I would heavily rely on AI to come up with a decent rating system.

The below is a rudimentary flow chart to show what my preliminary pipeline looked like for processing posts and comments, using LLMs. While I was working on the pipeline, I also set up a quick UI to view the data, and with some prompt tweaking and iterations.