Computer Vision API (v3.2-preview.3)

The Computer Vision API provides state-of-the-art algorithms to process images and return information. For example, it can be used to determine if an image contains mature content, or it can be used to find all the faces in an image. It also has other features like estimating dominant and accent colors, categorizing the content of images, and describing an image with complete English sentences. Additionally, it can also intelligently generate images thumbnails for displaying large images effectively.

This API is currently available in:

  • Australia East - australiaeast.api.cognitive.microsoft.com
  • Brazil South - brazilsouth.api.cognitive.microsoft.com
  • Canada Central - canadacentral.api.cognitive.microsoft.com
  • Central India - centralindia.api.cognitive.microsoft.com
  • Central US - centralus.api.cognitive.microsoft.com
  • East Asia - eastasia.api.cognitive.microsoft.com
  • East US - eastus.api.cognitive.microsoft.com
  • East US 2 - eastus2.api.cognitive.microsoft.com
  • France Central - francecentral.api.cognitive.microsoft.com
  • Japan East - japaneast.api.cognitive.microsoft.com
  • Japan West - japanwest.api.cognitive.microsoft.com
  • Korea Central - koreacentral.api.cognitive.microsoft.com
  • North Central US - northcentralus.api.cognitive.microsoft.com
  • North Europe - northeurope.api.cognitive.microsoft.com
  • South Africa North - southafricanorth.api.cognitive.microsoft.com
  • South Central US - southcentralus.api.cognitive.microsoft.com
  • Southeast Asia - southeastasia.api.cognitive.microsoft.com
  • UK South - uksouth.api.cognitive.microsoft.com
  • West Central US - westcentralus.api.cognitive.microsoft.com
  • West Europe - westeurope.api.cognitive.microsoft.com
  • West US - westus.api.cognitive.microsoft.com
  • West US 2 - westus2.api.cognitive.microsoft.com

Read

Use this call to perform a Read operation. The Read API is optimized for text-heavy images and multi-page, mixed language, and mixed type (print – sixty-three languages and handwritten – English only) documents. The Read operation executes asynchronously. When you call the Read operation, the call returns with a response header called 'Operation-Location'. The 'Operation-Location' header contains a URL with the Operation Id to be used in the second step. In the second step, you use the Get Read Result operation to fetch the detected text lines and words as part of the JSON response. The time for completion of the text extraction process depends on the volume of the text and the number of pages in the document.

Supported languages: The Read API is currently only available for Afrikaans, Albanian, Asturian, Basque, Bislama, Breton, Catalan, Cebuano, Cornish, Corsican, Czech, Danish, Dutch, English , Fijian, Filipino, French, Friulian, Galician, German, Gilbertese, Greenlandic, Haitian Creole, Hani, Hmong Daw, Hungarian, Indonesian, Interlingua, Inuktitut, Irish, Italian, Japanese, Javanese, Kabuverdianu, Kachin, Kara-Kalpak, Khasi, Korean, K’iche’, Luxembourgish, Malay, Manx, Neapolitan, Norwegian, Occitan, Polish, Portuguese, Romansh, Scots, Scottish Gaelic, simplified Chinese , Spanish, Swahili, Swedish, Tetum, traditional Chinese , Turkish, Uzbek, Volapük, Western Frisian, Yucatec Maya, Zhuang and Zulu.

Select the testing console in the region where you created your resource:

Open API testing console

Request URL

Request parameters

(optional)
string

The BCP-47 language code of the text in the document. Currently, only Afrikaans (‘af’), Albanian (‘sq’), Asturian (‘ast’), Basque (‘eu’), Bislama (‘bi’), Breton (‘br’), Catalan (‘ca’), Cebuano (‘ceb’), Chamorro (‘ch’), Cornish (‘kw’), Corsican (‘co’), Crimean Tatar - Latin script(‘crh’), Czech (‘cs’), Danish (‘da’), Dutch (‘nl’), English ('en'), Estonian (‘et’), Fijian (‘fj’), Filipino (‘fil’), Finnish (‘fi’), French (‘fr’), Friulian (‘fur’), Galician (‘gl’), German (‘de’), Gilbertese (‘gil’), Greenlandic (‘kl’), Haitian Creole (‘ht’), Hani (‘hni’), Hmong Daw (‘mww’), Hungarian (‘hu’), Indonesian (‘id’), Interlingua (‘ia’), Inuktitut (‘iu’), Irish (‘ga’), Italian (‘it’), Japanese (‘ja’), Javanese (‘jv’), Kabuverdianu (‘kea’), Kachin (‘kac’), Kara-Kalpak (‘kaa’), Kashubian (‘csb’), Khasi (‘kha’), Korean (‘ko’), Kurdish - Latin script (‘ku’), K’iche’ (‘quc’), Luxembourgish (‘lb’), Malay (‘ms’), Manx (‘gv’), Neapolitan (‘nap’), Norwegian (‘no’), Occitan (‘oc’), Polish (‘pl’), Portuguese (‘pt’), Romansh (‘rm’), Scots (‘sco’), Scottish Gaelic (‘gd’), simplified Chinese (‘zh-Hans’), Slovenian (‘sl’), Spanish (‘es’), Swahili (‘sw’), Swedish (‘sv’), Tatar - Latin script (‘tt’), Tetum (‘tet’), traditional Chinese (‘zh-Hant’), Turkish (‘tr’), Upper Sorbian (‘hsb’), Uzbek (‘uz’), Volapük (‘vo’), Walser (‘wae’), Western Frisian (‘fy’), Yucatec Maya (‘yua’), Zhuang (‘za’) and Zulu (‘zu’) are supported (print – seventy-three languages and handwritten – English only). Read supports auto language identification and multilanguage documents, so only provide a language code if you would like to force the documented to be processed as that specific language.

(optional)
string

The page selection only leveraged for multi-page PDF and TIFF documents. Accepted input include single pages (e.g.'1, 2' -> pages 1 and 2 will be processed), finite (e.g. '2-5' -> pages 2 to 5 will be processed) and open-ended ranges (e.g. '5-' -> all the pages from page 5 will be processed & e.g. '-10' -> pages 1 to 10 will be processed). All of these can be mixed together and ranges are allowed to overlap (eg. '-5, 1, 3, 5-10' - pages 1 to 10 will be processed). The service will accept the request if it can process at least one page of the document (e.g. using '5-100' on a 5 page document is a valid input where page 5 will be processed). If no page range is provided, the entire document will be processed.

(optional)
string

Optional parameter to specify which reading order algorithm should be applied when ordering the extract text elements. Can be either 'basic' or 'natural'. Will default to basic if not specified

Request headers

string
Media type of the body sent to the API.
string
Subscription key which provides access to this API. Found in your Cognitive Services accounts.

Request body

Input passed within the POST body. Supported input methods: raw image binary or image URL.

Input requirements:

  • Supported image formats: JPEG, PNG, BMP, PDF and TIFF.
  • Please do note MPO (Multi Picture Objects) embedded JPEG files are not supported.
  • For multi-page PDF and TIFF documents:
    • For the free tier, only the first 2 pages are processed.
    • For the paid tier, up to 2,000 pages are processed.
  • Image file size must be less than 50 MB (4 MB for the free tier).
  • The image/document page dimensions must be at least 50 x 50 pixels and at most 10000 x 10000 pixels.
  • The PDF file dimensions must be at most 17 x 17 inches, corresponding to Legal or A3 paper sizes and smaller.

{"url":"https://intelligentkioskstore.blob.core.windows.net/visionapi/suggestedphotos/3.png"}
{
  "type": "object",
  "required": [
    "url"
  ],
  "properties": {
    "url": {
      "description": "Publicly reachable URL of an image.",
      "type": "string"
    }
  }
}
[Binary image data]

Response 202

The service has accepted the request and will start processing soon. The client can query the operation status and result by using the Operation Id from the 'Operation-Location' response header value, a URL. In the following example, the Operation Id is 49a36324-fc4b-4387-aa06-090cfbf0064f, to be used as the ‘operationId’ parameter to the Get Read Results operation, The URL expires in 24 hours.

Operation-Location Operation status and result URL.
Example: https://cognitiveservice/vision/v3.2-preview.3/read/analyzeResults/49a36324-fc4b-4387-aa06-090cfbf0064f

Sample Response Header:

Content-Length: 0 Operation-Location: https://japaneast.api.cognitive.microsoft.com/vision/v3.2-preview.3/read/analyzeResults/08f557b9-c915-4c0d-a1ba-4648a4405195 CSP-Billing-Usage: CognitiveServices.ComputerVision.Transaction=1 apim-request-id: 08f557b9-c915-4c0d-a1ba-4648a4405195 Date: Fri, 04 Sep 2020 10:11:18 GMT

Response 400

Bad request. The error code and message are returned in the JSON:

Error Code Description
BadArgument Unsupported media type.
BadArgument Bad or unrecognizable request JSON or binary file.
InvalidImageURL Image URL is badly formatted.
FailedToDownloadImage Failed to download image from input URL.
InvalidImage The input data is not a valid image or password protected.
UnsupportedImageFormat Image format unsupported. Supported formats include JPEG, PNG, BMP, PDF and TIFF.
InvalidImageSize The input image is too large. It should not be larger than 50MB.
InvalidImageDimension The input image dimension is out of range. The minimum image/document page dimension is 50 x 50 pixels and the maximum is 10000 x 10000 pixels. The maximum PDF dimension is 17 x 17 inches.
InvalidReadingOrder The specified Reading Order is invalid.
InvalidPageRange The specified Pages parameter is invalid. Please see returned error for more details. 

{ 
    "error": {
        "code": "BadArgument", 
        "message": "Invalid input."
    }
}

Response 415

Unsupported media type. 'Content-Type' does not match the content of the POST request.

  • For image URL, 'Content-Type' should be application/json.
  • For binary image data, 'Content-Type' should be application/octet-stream.

{
    "error": {
        "code": "BadArgument",
        "message": "Unsupported media type."
    }
}

Response 500

Returns 500 if there's any internal error.

{
    "error": {
        "code": "Unspecified", 
        "message": "Internal server error."
    }
}

Response 503

Returns 503 if transient faults occurred when dealing with Microsoft Azure storage services.

{
    "error": {
        "code": "StorageException", 
        "message": "Transient faults occurred when dealing with Microsoft Azure storage services, please try it later."
    }
}

Code samples

@ECHO OFF

curl -v -X POST "https://virginia.api.cognitive.microsoft.us/vision/v3.2-preview.3/read/analyze?language={string}&pages={string}&readingOrder={string}"
-H "Content-Type: application/json"
-H "Ocp-Apim-Subscription-Key: {subscription key}"

--data-ascii "{body}" 

using System;
using System.Net.Http.Headers;
using System.Text;
using System.Net.Http;
using System.Web;

namespace CSHttpClientSample
{
    static class Program
    {
        static void Main()
        {
            MakeRequest();
            Console.WriteLine("Hit ENTER to exit...");
            Console.ReadLine();
        }
        
        static async void MakeRequest()
        {
            var client = new HttpClient();
            var queryString = HttpUtility.ParseQueryString(string.Empty);

            // Request headers
            client.DefaultRequestHeaders.Add("Ocp-Apim-Subscription-Key", "{subscription key}");

            // Request parameters
            queryString["language"] = "{string}";
            queryString["pages"] = "{string}";
            queryString["readingOrder"] = "{string}";
            var uri = "https://virginia.api.cognitive.microsoft.us/vision/v3.2-preview.3/read/analyze?" + queryString;

            HttpResponseMessage response;

            // Request body
            byte[] byteData = Encoding.UTF8.GetBytes("{body}");

            using (var content = new ByteArrayContent(byteData))
            {
               content.Headers.ContentType = new MediaTypeHeaderValue("< your content type, i.e. application/json >");
               response = await client.PostAsync(uri, content);
            }

        }
    }
}	
// // This sample uses the Apache HTTP client from HTTP Components (http://hc.apache.org/httpcomponents-client-ga/)
import java.net.URI;
import org.apache.http.HttpEntity;
import org.apache.http.HttpResponse;
import org.apache.http.client.HttpClient;
import org.apache.http.client.methods.HttpGet;
import org.apache.http.client.utils.URIBuilder;
import org.apache.http.impl.client.HttpClients;
import org.apache.http.util.EntityUtils;

public class JavaSample 
{
    public static void main(String[] args) 
    {
        HttpClient httpclient = HttpClients.createDefault();

        try
        {
            URIBuilder builder = new URIBuilder("https://virginia.api.cognitive.microsoft.us/vision/v3.2-preview.3/read/analyze");

            builder.setParameter("language", "{string}");
            builder.setParameter("pages", "{string}");
            builder.setParameter("readingOrder", "{string}");

            URI uri = builder.build();
            HttpPost request = new HttpPost(uri);
            request.setHeader("Content-Type", "application/json");
            request.setHeader("Ocp-Apim-Subscription-Key", "{subscription key}");


            // Request body
            StringEntity reqEntity = new StringEntity("{body}");
            request.setEntity(reqEntity);

            HttpResponse response = httpclient.execute(request);
            HttpEntity entity = response.getEntity();

            if (entity != null) 
            {
                System.out.println(EntityUtils.toString(entity));
            }
        }
        catch (Exception e)
        {
            System.out.println(e.getMessage());
        }
    }
}


<!DOCTYPE html>
<html>
<head>
    <title>JSSample</title>
    <script src="http://ajax.googleapis.com/ajax/libs/jquery/1.9.0/jquery.min.js"></script>
</head>
<body>

<script type="text/javascript">
    $(function() {
        var params = {
            // Request parameters
            "language": "{string}",
            "pages": "{string}",
            "readingOrder": "{string}",
        };
      
        $.ajax({
            url: "https://virginia.api.cognitive.microsoft.us/vision/v3.2-preview.3/read/analyze?" + $.param(params),
            beforeSend: function(xhrObj){
                // Request headers
                xhrObj.setRequestHeader("Content-Type","application/json");
                xhrObj.setRequestHeader("Ocp-Apim-Subscription-Key","{subscription key}");
            },
            type: "POST",
            // Request body
            data: "{body}",
        })
        .done(function(data) {
            alert("success");
        })
        .fail(function() {
            alert("error");
        });
    });
</script>
</body>
</html>

#import <Foundation/Foundation.h>

int main(int argc, const char * argv[])
{
    NSAutoreleasePool * pool = [[NSAutoreleasePool alloc] init];
    
    NSString* path = @"https://virginia.api.cognitive.microsoft.us/vision/v3.2-preview.3/read/analyze";
    NSArray* array = @[
                         // Request parameters
                         @"entities=true",
                         @"language={string}",
                         @"pages={string}",
                         @"readingOrder={string}",
                      ];
    
    NSString* string = [array componentsJoinedByString:@"&"];
    path = [path stringByAppendingFormat:@"?%@", string];

    NSLog(@"%@", path);

    NSMutableURLRequest* _request = [NSMutableURLRequest requestWithURL:[NSURL URLWithString:path]];
    [_request setHTTPMethod:@"POST"];
    // Request headers
    [_request setValue:@"application/json" forHTTPHeaderField:@"Content-Type"];
    [_request setValue:@"{subscription key}" forHTTPHeaderField:@"Ocp-Apim-Subscription-Key"];
    // Request body
    [_request setHTTPBody:[@"{body}" dataUsingEncoding:NSUTF8StringEncoding]];
    
    NSURLResponse *response = nil;
    NSError *error = nil;
    NSData* _connectionData = [NSURLConnection sendSynchronousRequest:_request returningResponse:&response error:&error];

    if (nil != error)
    {
        NSLog(@"Error: %@", error);
    }
    else
    {
        NSError* error = nil;
        NSMutableDictionary* json = nil;
        NSString* dataString = [[NSString alloc] initWithData:_connectionData encoding:NSUTF8StringEncoding];
        NSLog(@"%@", dataString);
        
        if (nil != _connectionData)
        {
            json = [NSJSONSerialization JSONObjectWithData:_connectionData options:NSJSONReadingMutableContainers error:&error];
        }
        
        if (error || !json)
        {
            NSLog(@"Could not parse loaded json with error:%@", error);
        }
        
        NSLog(@"%@", json);
        _connectionData = nil;
    }
    
    [pool drain];

    return 0;
}

<?php
// This sample uses the Apache HTTP client from HTTP Components (http://hc.apache.org/httpcomponents-client-ga/)
require_once 'HTTP/Request2.php';

$request = new Http_Request2('https://virginia.api.cognitive.microsoft.us/vision/v3.2-preview.3/read/analyze');
$url = $request->getUrl();

$headers = array(
    // Request headers
    'Content-Type' => 'application/json',
    'Ocp-Apim-Subscription-Key' => '{subscription key}',
);

$request->setHeader($headers);

$parameters = array(
    // Request parameters
    'language' => '{string}',
    'pages' => '{string}',
    'readingOrder' => '{string}',
);

$url->setQueryVariables($parameters);

$request->setMethod(HTTP_Request2::METHOD_POST);

// Request body
$request->setBody("{body}");

try
{
    $response = $request->send();
    echo $response->getBody();
}
catch (HttpException $ex)
{
    echo $ex;
}

?>
########### Python 2.7 #############
import httplib, urllib, base64

headers = {
    # Request headers
    'Content-Type': 'application/json',
    'Ocp-Apim-Subscription-Key': '{subscription key}',
}

params = urllib.urlencode({
    # Request parameters
    'language': '{string}',
    'pages': '{string}',
    'readingOrder': '{string}',
})

try:
    conn = httplib.HTTPSConnection('virginia.api.cognitive.microsoft.us')
    conn.request("POST", "/vision/v3.2-preview.3/read/analyze?%s" % params, "{body}", headers)
    response = conn.getresponse()
    data = response.read()
    print(data)
    conn.close()
except Exception as e:
    print("[Errno {0}] {1}".format(e.errno, e.strerror))

####################################

########### Python 3.2 #############
import http.client, urllib.request, urllib.parse, urllib.error, base64

headers = {
    # Request headers
    'Content-Type': 'application/json',
    'Ocp-Apim-Subscription-Key': '{subscription key}',
}

params = urllib.parse.urlencode({
    # Request parameters
    'language': '{string}',
    'pages': '{string}',
    'readingOrder': '{string}',
})

try:
    conn = http.client.HTTPSConnection('virginia.api.cognitive.microsoft.us')
    conn.request("POST", "/vision/v3.2-preview.3/read/analyze?%s" % params, "{body}", headers)
    response = conn.getresponse()
    data = response.read()
    print(data)
    conn.close()
except Exception as e:
    print("[Errno {0}] {1}".format(e.errno, e.strerror))

####################################
require 'net/http'

uri = URI('https://virginia.api.cognitive.microsoft.us/vision/v3.2-preview.3/read/analyze')
uri.query = URI.encode_www_form({
    # Request parameters
    'language' => '{string}',
    'pages' => '{string}',
    'readingOrder' => '{string}'
})

request = Net::HTTP::Post.new(uri.request_uri)
# Request headers
request['Content-Type'] = 'application/json'
# Request headers
request['Ocp-Apim-Subscription-Key'] = '{subscription key}'
# Request body
request.body = "{body}"

response = Net::HTTP.start(uri.host, uri.port, :use_ssl => uri.scheme == 'https') do |http|
    http.request(request)
end

puts response.body