Unable to get position of text-insertion caret from AX when no text is selected

Originator:peter.kamb
Number:rdar://14285519 Date Originated:June 26 2013
Status: Resolved:
Product:Accessibility Product Version:
Classification: Reproducible:
 
Summary:

It is important for accessibility tools to know the on-screen position of the text-insertion caret in the focused textfield.

A tool for low visiiblity users could, for example, show a zoomed-in version of the word being typed. This helper window should appear next to the word; the position would be determined via the position of the text-insertion point.

When text is selected (highlighted), it is easy to get the bounds of the entire selected area with the accessibilityAPI's `kAXBoundsForRangeParameterizedAttribute`. The caret position can then be determined from this bounding rectangle.

However, when text is NOT selected, attempting to get the value of the `kAXBoundsForRangeParameterizedAttribute` attribute returns `kAXErrorNoValue`. Apple docs state this error code means "The parameterized attribute is not supported by the AXUIElementRef".

Getting the value of this attribute, when no text is selected, should instead return the same bounding rectangle as  when text *is* selected. However the length of that rect should be 0 (or perhaps the tiny width of the caret itself). Accessibility tools would then be able to calculate the text-insertion caret position without first "faking" a selection.

Steps to Reproduce:

The below code can be used to test a textfield.

Put your text-insertion caret in a textfield. Then run the selectionRect method.

    - (AXUIElementRef)focusedApp {
        pid_t pid;
        ProcessSerialNumber psn;
        GetFrontProcess(&psn);
        GetProcessPID(&psn, &pid);
        AXUIElementRef focusedApp = AXUIElementCreateApplication(pid);
        
        return focusedApp;
    }
    
    - (AXUIElementRef)focusedElement
    {
        AXUIElementRef focusedApp = [self focusedApp];
        
        AXUIElementRef focusedElement;
        AXError focusedElementError = AXUIElementCopyAttributeValue(focusedApp, kAXFocusedUIElementAttribute, (CFTypeRef *)&focusedElement);
        if (focusedElementError == kAXErrorSuccess) {
            return focusedElement;
        }
        else {
            return nil;
        }
    }
    
    - (CGRect)selectionRect
    {
        AXUIElementRef focusedElement = [self focusedElement];
        
        AXValueRef selectionRangeValue;
        AXError selectionRangeError = AXUIElementCopyAttributeValue(focusedElement, kAXSelectedTextRangeAttribute, (CFTypeRef *)&selectionRangeValue);
        if (selectionRangeError == kAXErrorSuccess)
        {
            CFRange selectionRange;
            AXValueGetValue(selectionRangeValue, kAXValueCFRangeType, &selectionRange);
            
            //selectionRange.length is 0 for "no selection" (aka a bare caret insertion point)
            NSLog(@"Range: %lu, %lu", selectionRange.length, selectionRange.location);
            
            AXValueRef selectionBoundsValue;
            AXError selectionBoundsError = AXUIElementCopyParameterizedAttributeValue(focusedElement, kAXBoundsForRangeParameterizedAttribute, selectionRangeValue, (CFTypeRef *)&selectionBoundsValue);
            
            if (selectionRange.length == 0 && selectionBoundsError == kAXErrorSuccess) {
                NSLog(@"This works in TextMate 2, but nowhere else that I have seen.");
                
                NSLog(@"This case is the objective of this bug report");
            }
            
            if (selectionRange.length > 0 && selectionBoundsError == kAXErrorSuccess) {
                NSLog(@"It's easy to get the selection bounds rect when text is selected.");
            }
            
            if (selectionBoundsError == kAXErrorSuccess)
            {
                CGRect selectionBounds;
                AXValueGetValue(selectionBoundsValue, kAXValueCGRectType, &selectionBounds);
                
                NSLog(@"This will generally only work if text is highlighted");
                NSLog(@"Selection rect: (%f, %f) (%f, %f)", selectionBounds.origin.x, selectionBounds.origin.y, selectionBounds.size.width, selectionBounds.size.height);
                
                return selectionBounds;
            }
            else if (selectionBoundsError == kAXErrorNoValue)
            {
                NSLog(@"Could not get selection rect. SelectionRange.length == %lu", selectionRange.length);
                return CGRectMake(0, 0, 0, 0);
            }
        }
        
        return CGRectMake(0, 0, 0, 0);
    }

Expected Results:

The current textfield's `kAXBoundsForRangeParameterizedAttribute` should always get the on-screen rect of the current selection.

The text-insertion caret position is, esentially, a 0-length selection.

Thus `kAXBoundsForRangeParameterizedAttribute` should return a 0-length rect when no text is selected.

Importantly, though, the rect should always contain the origin and height of the (0-length) selection!

When one character is selected:
 - returns CGRect(100, 300, 6, 15)

When 0 characters are selected
 - should return CGRect(100, 300, 0, 15)
 - actually returns `kAXErrorNoValue`!
 - in the code above, we fall back to CGRectMake(0, 0, 0, 0)

Actual Results:

1. Select some text in the textfield.
2. Run the code
3. Notice that the code returns the bounding rect of the highlighted text.
4. The caret position can be calculated as the left or right edge of the bounding rect.

- vs -

1. Put the text-insertion caret in the text, but **do not** select any characters. Standard insertion caret.
2. Run the code
3. Attempting to get the bounding rect of the "selection" returns `kAXErrorNoValue`
4. No bounding rect can be calculated.
5. Thus the position of the caret cannot be found. This code returns CGRectMake(0, 0, 0, 0).

Regression:

Unknown. Behaves this way in 10.6, 10.7, 10.8

Notes:

In the vast majority of OS X text fields, in both Apple and Third Party apps, this bug report is true. Getting the `kAXBoundsForRangeParameterizedAttribute` attribute value fails with `kAXErrorNoValue` when the selection is empty.

However, I have seen that in the app TextMate 2 (2.0-alpha.9427) it actually works. You can get the frame from the bare text insertion point caret, no selection needed. This is the only app or textfield where I have observed this to work.

In TextMate 2, the reported no-selection bounds rect is CGRect(1461, 1302, -1, 16)

Note the "-1" as the width, not 0.

I have not looked into why TextMate is reporting this value. But it is very useful. You can get the text insertion point position at any time, with or without a selection.

Apple's apps and the standard cocoa controls should allow access as well.

Comments


Please note: Reports posted here will not necessarily be seen by Apple. All problems should be submitted at bugreport.apple.com before they are posted here. Please only post information for Radars that you have filed yourself, and please do not include Apple confidential information in your posts. Thank you!