chat({ content: [ { type: "image", base64: base64Data, mimeType: mediaTypeMap[imageType] || "image/png", }, { type: "text", content: dedent` Analyze this screenshot of a recipe webpage and extract recipe information. - Extract all available information visible in the screenshot. - If something is not found, use null for optional fields. - For ingredients, parse quantities and units. - Always provide heuristicAmount + heuristicUnit when possible, and also provide measuredAmount + measuredUnit when possible. - (e.g., "2 cups flour" -> measuredAmount: 250, measuredUnit: 'g', heuristicAmount: 2, heuristicUnit: 'cups', name: "flour") - Do use the same values for heuristicAmount and measuredAmount. - In most cases a recipe will have heuristicAmount and we should guess a measuredAmount - If the ingredient name contains some preparation (i.e melted, beaten, etc.), remove it from the name and add it to the notes. (e.g., "2 cups melted butter" -> measuredAmount: 250, measuredUnit: 'g', notes: "melted", name: "butter") - For "group": If ingredients are organized into groups (e.g., "For the sauce:", "For the pasta:", "For the marinade:"), include the group name in the "group" field. If ingredients are not grouped, set it to null. - Represent tablespoon as tbsp and teaspoon as tsp. - Convert all times to minutes and output numbers (not strings) for numeric fields. - Generate a description if one is not provided. - For "source": Try to identify the source name. Use null if unknown. - For "url": Always include the source URL (${url}) here. `, }, ], outputSchema: UrlRecipeExtractionSchema, maxTokens: 4096, adapter, modelName, });