The @dynamicMemberLookup attribute was introduced in Swift 4.2 to add a certain degree of dynamism into the language similar to what is seen in languages like Python. When applied to a type, properties from this type will be resolved in runtime, meaning that you can call things that don't necessarily exist like they were explicitly defined:

@dynamicMemberLookup class MyClass {
    subscript(dynamicMember input: String) -> String {
	      return input == "foo" ? "bar" : "SwiftRocks"
    }
}

MyClass().foo // bar
MyClass().notFoo // SwiftRocks
// These properties don't exist, but they can be called because the type is @dynamicMemberLookup.

As seen in the snippet, usage of this attribute instead forces the type to provide a dynamicMember subscript which receives the "fake" property name as a parameter and acts upon it.

The original motivation was that it could be used in interoperability layers with dynamic languages, allowing you to call Python code in Swift like you would in Python itself, for example. Although this attribute doesn't have pure Swift in mind, you can certainly use it for it. I would probably never use it in regular iOS development, but my favorite use case is being able to improve JSON parsing:

let data: String? = dict["data"] as? String
//
let data: String? = dict.data //dynamically search for "data"

In a similar fashion, @dynamicCallable was added in Swift 5.0 as a follow-up to add the ability to dynamically call a method from a dynamicCallable type:

let myType: MyDynamicType = MyDynamicType()
myType(someArg2: someVal, someArg2: someVal2)

I was interested in learning how the attributes worked inside the compiler to know more about how they are able to transform these fake expressions into legit ones, so I've once again reverse-engineered the Swift compiler to find these answers - and used this knowledge to create my own Swift attribute.

Disclaimer: As always, this is a result of my own research and reverse-engineering for the pure purpose of learning something new. As I obviously have nothing to do with the original development of these attributes, some assumptions might not be fully correct. Feel free to correct me if you know how the compiler works!

This article will focus on the internals of @dynamicMemberLookup. @dynamicCallable works a bit differently, but overall follows the same idea.

Attribute Declaration/Parsing

A quick repo search shows us that in Swift, all attributes are mainly defined in the Attr.def file. Here's the definition of @dynamicMemberLookup:

SIMPLE_DECL_ATTR(dynamicMemberLookup, DynamicMemberLookup,
OnNominalType, 9)

A "simple attribute" is an attribute that holds no data (unlike @available which contains arguments). The first argument here is the name of the attribute in source files, the second one is the name of the attribute to be used inside the compiler (which will resolve to DynamicMemberLookupAttr), the third one defines its scope (in this case, a NominalType refers to the congregate of classes, structs, enums and protocols), and the last one is an unique code used internally to determine a valid attribute.

The first contact the Swift compiler has with attributes is during parsing. After lexical analysis generates the tokenized version of your code, the Parser will step through these tokens in order to generate a basic Abstract Syntax Tree (your code in a tree-like structure) which will later be used to further "understand" your code once typechecking is performed. This process can fail if it finds something it didn't expect - a mistyped keyword, an attribute in the wrong place, and so on. You have certainly seen this before in the shape of a "expected X identifier" error.

You can tell the compiler to print the Parser's resulting AST by running swiftc -dump-parse, but unfortunately for article purposes that doesn't print a type's attributes (a pull request opportunity?). But the good news is that we can confirm this by looking at the code that parses declaractions. The Parser does a gigantic amount of things, so I'll cheery-pick the relevant backtrace:

ParseStmt.cpp:387-391 - if the current token represents a declaration, attempt to parse it.

if isStartOfDecl()
  parseDecl(...)

ParseDecl.cpp:2639-2710 - For a regular declaraction expression the attributes should be the first tokens, so they are the first tokens to be parsed:

Parser::parseDecl(ParseDeclOptions Flags, llvm::function_ref<void(Decl*)> Handler) {
    //removed: dealing with #if/#warning/#error
    parseDeclAttributeList(...)
    //rest of the parsing for a declaration
}

What parseDeclAttributeList() does is a do while loop to parse an attribute if the current token is an @, which then calls parseDeclAttribute() to begin parsing the attribute:

bool Parser::parseDeclAttribute(DeclAttributes &Attributes, SourceLoc AtLoc)
    if (Tok.isNot(tok::identifier) &&
        Tok.isNot(tok::kw_in) &&
        Tok.isNot(tok::kw_inout)) {
        diagnose(Tok, diag::expected_attribute_name); // Compiler error for "Expected attribute name"
        return true;
    }  
    DeclAttrKind DK = DeclAttribute::getAttrKindFromString(Tok.getText());

    // FIXME: This renaming happened before Swift 3, we can probably remove
    // the specific fallback path at some point.
    checkInvalidAttrName("availability", "available", DAK_Available, diag::attr_renamed); // Checks if the attribute name matches the old name and fails, suggesting the new
    // more checks for all renamed or deprecated attributes

    if (DK == "a valid attribute from Attr.def") // line 1805
        return parseNewDeclAttribute(Attributes, AtLoc, DK);
    diagnose(Tok, diag::unknown_attribute, Tok.getText()); // Compilation error for "unknown attribute %@"
}

I like this method because we can see how Swift treats renamed attributes - just explicitely check if the current token matches an old name and throw an error stating that it's now called something else. But in short, we're just seeing if the name of our attribute matches an attribute defined at Attr.def, halting compilation if that's not the case. If the attribute exists, parseNewDeclAttribute will consume the token and add it to an attributes list for that AST.

By running the Swift compiler with the -dump-parse attribute, we'll tell the compiler to start compiling but stop as soon as the parsing step ends. This allows us to confirm that this is indeed where this logic is being executed:

swiftc -dump-parse attrs.swift

@swiftRocks class Foo {} // error: unknown attribute 'swiftRocks'
@availability class Foo {} // error: '@availability' has been renamed to '@available'

Intermission: Creating a new @swiftRocks attribute

Before seeing how this attribute results in dynamic members, how about using this knowledge to actually make an attribute of our own?

This brief introduction shows us that the barebones of an attribute aren't that complicated at all, and we can use that information to create a basic @swiftRocks attribute.

For that, I'll just add an entry for a class attribute in Attr.def:

SIMPLE_DECL_ATTR(swiftRocks, SwiftRocks, OnClass, 83)

Doing so forced me to add my attribute to a few lists and add a visitSwiftRocksAttr() method in TypeCheckAttr.cpp, which I did but left it empty since my attribute does nothing at the moment:

void AttributeChecker::
visitSwiftRocksAttr(SwiftRocksAttr *attr) {}

This is enough to make a @swiftRocks type compile, although nothing will happen since there's no logic tied to it. To see something happen, I'll pretend that older Swift versions used this very useful attribute as @rockingSwift by adding a new check at parseDeclAttribute:

checkInvalidAttrName("rockingSwift", "swiftRocks", DAK_SwiftRocks, diag::attr_renamed);

@rockingSwift class Foo {} //error: '@rockingSwift' has been renamed to '@swiftRocks'

We'll get back to it later.

Defining/changing behaviour based on attributes

After parsing, @dynamicMemberLookup will come to play again during semantic analysis. In order to confirm that your code is legit, the compiler will annotate the AST's nodes with their respective types and confirm that they can do whatever it is that they are doing. Some debugging revealed that the typechecking of a declaration triggers a typechecking call for every attribute it contains - first to confirm that the attribute is on the correct type (in this case, a NominalType), and second in order for you to confirm that the attribute is being used correctly. The latter happens in the same place where I had to create my visitSwiftRocksAttr method, but in visitDynamicMemberLookupAttr instead. In short, this method checks if the type implements one or more valid subscript(dynamicMember), and throws a compilation error if that's not the case:

void AttributeChecker::
visitDynamicMemberLookupAttr(DynamicMemberLookupAttr *attr) {
    // This attribute is only allowed on nominal types.
    auto decl = cast<NominalTypeDecl>(D);
    auto type = decl->getDeclaredType();

    // Look up `subscript(dynamicMember:)` candidates.
    auto subscriptName = DeclName(TC.Context, DeclBaseName::createSubscript(),
                                  TC.Context.Id_dynamicMember);
    auto candidates = TC.lookupMember(decl, type, subscriptName);

    // If there are no candidates, then the attribute is invalid.
    if (candidates.empty()) {
      TC.diagnose(attr->getLocation(), diag::invalid_dynamic_member_lookup_type,
                  type);
      attr->setInvalid();
      return;
    }

    // If no candidates are valid, then reject one.
    auto oneCandidate = candidates.front();
    candidates.filter([&](LookupResultEntry entry, bool isOuter) -> bool {
      auto cand = cast<SubscriptDecl>(entry.getValueDecl());
      TC.validateDeclForNameLookup(cand);
      return isValidDynamicMemberLookupSubscript(cand, decl, TC);
    });

    if (candidates.empty()) {
      TC.diagnose(oneCandidate.getValueDecl()->getLoc(),
                  diag::invalid_dynamic_member_lookup_type, type);
      attr->setInvalid();
    }
}

As far as developing attributes goes, the standard seems to end here. Because attributes can be used for virtually anything, each attribute is developed where it makes sense for it. In @dynamicMemberLookup's case, this happens during semantic analysis - when the Constraint System fails to resolve our unexistent properties through normal means, checking the existence of this attribute is used as a last resort: (simplified for readability purposes, original method here)

MemberLookupResult ConstraintSystem::
performMemberLookup(...) {
  //Removed: Attempt resolve member through several means, but fail since the property doesn't exist

  // If we're about to fail lookup, but we are looking for members in a type
  // with the @dynamicMemberLookup attribute, then we resolve a reference
  // to a `subscript(dynamicMember:)` method and pass the member name as a
  // string parameter.
  if (cantResolveIt && isSimpleName) {
    auto name = memberName.getBaseIdentifier();
    if (hasDynamicMemberLookupAttribute(...)) {
      auto &ctx = getASTContext();
      // Find `subscript(dynamicMember:)` methods in this type.
      auto subscriptName = DeclName(ctx, DeclBaseName::createSubscript(), ctx.Id_dynamicMember);
      auto subscripts = performMemberLookup(constraintKind,
                                            subscriptName,
                                            baseTy, functionRefKind,
                                            memberLocator,
                                            includeInaccessibleMembers);
      for (auto candidate : subscripts.ViableCandidates) {
        auto decl = cast<SubscriptDecl>(candidate.getDecl());
        if (isValidDynamicMemberLookupSubscript(decl, DC, TC))
          result.addViable(OverloadChoice::getDynamicMemberLookup(baseTy, decl, name));
      }
  }
}

By confirming that the fake property comes from a type that uses the attribute (remember that the attribute was added to the declaration's AST), the solver concludes that it's possible to resolve it by overloading it with the type's subscript(dynamicMember:) declaration.

After the CS resolves the intended return type of the property, the Sema's Solution Application phase will detect the desired overload solution and generate a subscript expression that matches the original definition inside the type. Finally, this expression replaces the original property call. (original file here)

case OverloadChoiceKind::DynamicMemberLookup: {
  // Application of a DynamicMemberLookup result turns a member access of
  // x.foo into x[dynamicMember: "foo"].
 
  // Removed for readability

  // Generate a (dynamicMember: T) expression.
  auto fieldName = selected.choice.getName().getBaseIdentifier().str();
  auto index = buildDynamicMemberLookupIndexExpr(fieldName, ...);

  // Build and return a subscript that uses this string as the index.
  return buildSubscript(base, index, ctx.Id_dynamicMember, ...)
}

As spoiled by the comment above, this means that @dynamicMemberLookup properties are just syntax sugars for subscript calls! Because our fake properties really don't exist, the compiler swaps them with calls to the subscript method required by the attribute.

You can confirm this by compiling with the -dump-ast argument. Similar to -dump-parse, this argument will stop the compilation after typechecking is performed, allowing you to see the complete version of the AST. For let foo: String = myType.bar, the result will be something like this:

(pattern_named type='String' 'foo')
 (subscript_expr type='String'
  (tuple_expr implicit type='(dynamicMember: String)' names=dynamicMember
   (string_literal_expr implicit type='String' value="bar")))

...which vaguely means let foo: String = myType[dynamicMember: "bar"].

Bonus: Adding functionality to @swiftRocks

Adding requirements

Now that @dynamicMemberLookup is uncovered, we're ready to make our custom attribute actually do something.

The first thing I want to change is the checker function I had to add when the attribute was created. I want this attribute to work only in classes that are called ClassThatRocks. If that's not the case, compilation must fail.

To be able to do that, I added a new identifier called id_ClassThatRocks to the compiler's list of known identifiers, and a "not ClassThatRocks" error to the compiler's list of semantic analysis related errors:

ERROR(invalid_swiftrocks_name,none,
"@swiftRocks requires %0 to be called 'ClassThatRocks'", (Type))

With that in place, I just need to compare the declaration's name in visitSwiftRocksAttr():

void AttributeChecker::
visitSwiftRocksAttr(SwiftRocksAttr *attr) {
  auto decl = cast<NominalTypeDecl>(D);
  auto type = decl->getDeclaredType();
  if (decl->getName() != TC.Context.Id_ClassThatRocks) {
    TC.diagnose(attr->getLocation(),
                diag::invalid_swiftrocks_name, type);
    attr->setInvalid();
  }
}

And the result is:

@swiftRocks class Foo {} //error: @swiftRocks requires 'Foo' to be called 'ClassThatRocks'
@swiftRocks class ClassThatRocks {} //Works!

Adding the actual functionality: Wholesome reminders

For the actual use, I've thought that such an incredible attribute should have an equally incredible use: When applied to a type, the compiler will put a warning on all properties that don't have "ThatRocks" in their name, because they are doing a good job and deserve recognition.

To do this, I'll intercept the typechecker in order to have access to all getter declarations. Given a getter, I can recursively its parents to see if someone has a @swiftRocks attribute and check if the getter's name doesn't contain "ThatRocks" in order to send the coder a friendly warning.

After a very long time of searching for suitable places for this implementation, I've found that typeCheckDecl() has all the information I need. It's probably a terrible place to do this, but the members of SwiftRocks unanimously decided that this attribute is more important than coding practices. After another very long time of trying to figure out how to retrieve a getter's "type tree", here's what I ended up with

void TypeChecker::typeCheckDecl(Decl *D) {
  if (auto AD = dyn_cast<AccessorDecl>(D)) {
    DeclName name = AD->getStorage()->getFullName();
    if (auto nominal = D->getDeclContext()->getSelfNominalTypeDecl()) {
      auto type = nominal->getDeclaredType();
      if (name.isSimpleName() && !name.isSpecial() && hasSwiftRocksAttribute(type)) {
        StringRef rocks = "ThatRocks";
        StringRef strName = name.getBaseIdentifier().str();
        if (!strName.contains(rocks)) {
          diagnose(AD->getLoc(), 
                   diag::invalid_swiftrocks_property_name,
                   strName);
        }
      }
    }
  }
  //removed: rest of the method
}

I'll spare you the details of hasSwiftRocksAttribute() because I just copied hasDynamicMemberLookupAttribute() and changed the attribute name, but it checks a type's parents until it find the attribute. Here's the original one if you're curious.

After building the compiler and running the following snippet, all properties of AwesomeClass get their hardwork recognized!

@swiftRocks class AwesomeClass {
    let number: Int = 1  //warning: Property 'number' is doing its best. Consider naming it 'numberThatRocks'.
    let stringThatRocks: String = "stringy"
}

Conclusion

I enjoy researching these features because they tell you a lot about how the language works. In this case, we can see that attributes have infinite possibilities - from stupid name checks to making properties pop from thin air. One might argue that they aren't "swifty" compared to the rest of the language, but they'll likely continue to be an integral part of the language for years to come.

Follow me on my Twitter - @rockbruno_, and let me know of any suggestions and corrections you want to share.

References and Good reads

SE-0195 - @dynamicMemberLookup
Original implementation of SE-0195
Typechecker Docs
The Swift Source Code