Einstein OCR provides OCR (optical character recognition) models that detect alphanumeric text in an image. In other words, Optical character recognition or optical character reader (OCR) is the electronic or mechanical conversion of images of typed, handwritten, or printed text into machine-encoded text, whether from a scanned document, or a photo of a document, a scene photo.
We have previously covered how we can use Einstein OCR to Extract Data from images, and tables. Today we will check how we can extract data using Invoices.
Generate Token for EInstein OCR
To make an API request, Firstly we need to generate an Access token. So you can find detailed steps for generating token here. Then we will make an API request to get data from the invoice. As this is beta so it provides few attributes and not all of the data.
The Einstein OCR invoice solution currently supports the list of entities below.
| Entity Name | Key Variation Examples |
|---|---|
invoice_number | invoice number, invoice #, invoice, invoice ID… |
invoice_date | invoice date, date, … |
due_date | due date, due on, … |
purchase_order | PO number, PO#, purchase order, … |
total_amount | total, total amount, … |
total_tax_amount | tax, total tax, … |
amount_due | due, amount due, … |
Extract Data using Einstein OCR
While it misses a few fields eg: addresses, it will cover most of the data we need from an invoice. So below is the sample code we have used.
public static void einsteinOCRInvoice() {
String access_token = getAccessToken();
HttpRequest req = new HttpRequest();
req.setMethod('POST');
req.setEndpoint('https://api.einstein.ai/v2/vision/ocr');
req.setHeader('content-type', 'multipart/form-data; charset="UTF-8"; boundary="1ff13444ed8140c7a32fc4e6451aa76d"');
req.setHeader('Authorization', 'Bearer '+access_token); //replace token with your access token
req.setHeader('Cache-Control', 'no-cache');
string form64 = '';
form64 += HttpFormBuilder.WriteBoundary();
form64 += HttpFormBuilder.WriteBodyParameter('sampleLocation', 'https://newstechnologystuff.com/wp-content/uploads/2022/06/SampleInvoice-724x1024.png');
form64 += HttpFormBuilder.WriteBoundary();
form64 += HttpFormBuilder.WriteBodyParameter('modelId', 'tabulatev2');
form64 += HttpFormBuilder.WriteBoundary();
form64 += HttpFormBuilder.WriteBodyParameter('task', 'invoice ');
form64 += HttpFormBuilder.WriteBoundary(HttpFormBuilder.EndingType.CrLf);
blob formBlob = EncodingUtil.base64Decode(form64);
string contentLength = string.valueOf(formBlob.size());
req.setBodyAsBlob(formBlob);
req.setHeader('Connection', 'keep-alive');
req.setHeader('Content-Length', contentLength);
req.setTimeout(60*1000);
Http h = new Http();
String resp;
HttpResponse res = h.send(req);
system.debug(res +'========='+res.getBody());
}We need to pass a few parameters here:
- sample location: it stores the location where our file has been stored.
- modelId: tabulatev2
- task: What type of data we are sending, invoice
When we send the request in the response we get the JSON. Which has all the extracted data from Einstein OCR. You can easily write your own JSON wrapper to parse data in apex.
{
"object": "predictresponse",
"probabilities": [
{
"probability": 0.9999801,
"label": "key-value",
"boundingBox": {
"minX": 628,
"minY": 229,
"maxX": 670,
"maxY": 242
},
"attributes": {
"blockId": 1,
"key": {
"entity": "invoice_number",
"boundingBox": {
"minX": 1,
"minY": 1,
"maxX": 1,
"maxY": 1
}
},
"value": {
"text": "US-001",
"boundingBox": {
"minX": 628,
"minY": 229,
"maxX": 670,
"maxY": 242
}
},
"pageNumber": "1"
}
},
{
"probability": 0.9999815,
"label": "key-value",
"boundingBox": {
"minX": 611,
"minY": 256,
"maxX": 670,
"maxY": 265
},
"attributes": {
"blockId": 2,
"key": {
"entity": "invoice_date",
"boundingBox": {
"minX": 1,
"minY": 1,
"maxX": 1,
"maxY": 1
}
},
"value": {
"text": "11/02/2019",
"boundingBox": {
"minX": 611,
"minY": 256,
"maxX": 670,
"maxY": 265
}
},
"pageNumber": "1"
}
},
{
"probability": 0.99997973,
"label": "key-value",
"boundingBox": {
"minX": 614,
"minY": 281,
"maxX": 670,
"maxY": 291
},
"attributes": {
"blockId": 3,
"key": {
"entity": "purchase_order",
"boundingBox": {
"minX": 1,
"minY": 1,
"maxX": 1,
"maxY": 1
}
},
"value": {
"text": "2312/2019",
"boundingBox": {
"minX": 614,
"minY": 281,
"maxX": 670,
"maxY": 291
}
},
"pageNumber": "1"
}
},
{
"probability": 0.99997747,
"label": "key-value",
"boundingBox": {
"minX": 610,
"minY": 306,
"maxX": 670,
"maxY": 316
},
"attributes": {
"blockId": 4,
"key": {
"entity": "due_date",
"boundingBox": {
"minX": 1,
"minY": 1,
"maxX": 1,
"maxY": 1
}
},
"value": {
"text": "26/02/2019",
"boundingBox": {
"minX": 610,
"minY": 306,
"maxX": 670,
"maxY": 316
}
},
"pageNumber": "1"
}
},
{
"probability": 0.999969,
"label": "key-value",
"boundingBox": {
"minX": 546,
"minY": 373,
"maxX": 666,
"maxY": 400
},
"attributes": {
"blockId": 5,
"key": {
"entity": "total_amount",
"boundingBox": {
"minX": 1,
"minY": 1,
"maxX": 1,
"maxY": 1
}
},
"value": {
"text": "$154.06",
"boundingBox": {
"minX": 546,
"minY": 373,
"maxX": 666,
"maxY": 400
}
},
"pageNumber": "1"
}
},
{
"probability": 0.99990964,
"label": "key-value",
"boundingBox": {
"minX": 647,
"minY": 618,
"maxX": 672,
"maxY": 631
},
"attributes": {
"blockId": 6,
"key": {
"entity": "total_tax_amount",
"boundingBox": {
"minX": 1,
"minY": 1,
"maxX": 1,
"maxY": 1
}
},
"value": {
"text": "9.06",
"boundingBox": {
"minX": 647,
"minY": 618,
"maxX": 672,
"maxY": 631
}
},
"pageNumber": "1"
}
},
{
"probability": 1.388243e-7,
"label": "key-value",
"boundingBox": {
"minX": 641,
"minY": 550,
"maxX": 671,
"maxY": 560
},
"attributes": {
"blockId": 7,
"key": {
"entity": "amount_due",
"boundingBox": {
"minX": 1,
"minY": 1,
"maxX": 1,
"maxY": 1
}
},
"value": {
"text": "15.00",
"boundingBox": {
"minX": 641,
"minY": 550,
"maxX": 671,
"maxY": 560
}
},
"pageNumber": "1"
}
}
],
"task": "invoice"
}You can find the complete code here. So using Einstein OCR we can Extract Data from PDFs, images, and other forms. With each release, we are getting new features in Einstein. What would you like to add to Einstein OCR, let me know in the comments? Happy Programming 🙂