top of page
Writer's picturefutureprooftechskills

Is ChatGPT+ ready for Mule 4?

On March 13, 2022, OpenAI shocked the world by releasing GPT-4 through a live event streamed on YouTube. The company also released an optimized version of GPT-3.5. Now both large language models (LLMs) and the legacy (previous) LLM are available on ChatGPT. In this post, I introduce the three models currently available on ChatGPT Plus, five prompts that I used to play around with the new models, and the strides made by OpenAI in teaching ChatGPT Mule 4, since my observations on Feb 21, 2023. Check out my previous post on my comments regarding MuleSoft development on GPT 3.5 (now Legacy) model.


Cover image for blog post. It reads, "Is ChatGPT Plus ready for Mule 4?" Graphic contains ChatGPT logo and XML snippet from experiment conversation.
Cover image

Quick Navigation


Legal Disclaimer: This post consists of experiments, conclusions, and opinions that originate with me and are not affiliated with my current employer at the time of this post, Salesforce and MuleSoft.


Differences between OpenAI GPT models

Before I dive deep, these three available models are available in ChatGPT.

Model

Description

​Legacy 3.5

Screenshot from ChatGPT Plus application of legacy model description that reads,   Legacy (GPT-3.5). The previous ChatGPT Plus model. Reasoning - 3 out of 5, Speed - 2 out of 5, Conciseness - 1 out of 5.
OpenAI's Legacy (GPT-3.5) model description

​Default (GPT-3.5)


Screenshot from ChatGPT Plus application of Default (GPT-3.5) model description that reads, Optimized for speed, currently available to Plus users. Reasoning - 3 out of 5, Speed - 5 out of 5, Conciseness - 2 out of 5.
OpenAI's Default (GPT-3.5) model description

​GPT-4


Screenshot from ChatGPT Plus application of model description that reads, GPT-4. Our most advanced model, available to Plus subscribers. GPT-4 excels at tasks that require advanced reasoning, complex instruction understanding, and more creativity.  Reasoning - 5 out of 5, Speed - 2 out of 5, Conciseness - 4 out of 5.
OpenAI's GPT-4 model description

First Impressions Video

Check out this embedded first impressions video to view my reaction as I played with the latest ChatGPT release.

Experiment


Prompts

While experimenting with the latest release, I used the following prompts and compared the outputs between GPT 3.5 Optimized and GPT 4. These prompts represent typical tasks done by MuleSoft developers.

Prompt No.

Prompt

1

​Build a Mule 4 flow that prints “Hello World”

2

Generate an MUnit test case for this code

3*

​What Mule 4 scope will allow me to process flows asynchronously?

4*

Build a Mule 4 flow that listens to an HTTP GET request on port 8082 with a query parameter that takes a name and returns a JSON string of “Hello” concatenated with the value of the “name” query parameter.

​5*

​Write the Mule 4 code to enable CSV streaming on an SFTP connector.

* Prompts with * will not be discussed in this post.


Prompt 1: Build a Mule 4 flow that prints Hello World


Default (GPT 3.5) -Optimized

In this graphic, I ask ChatGPT's Default (3.5) - optimized model to Build a Mule 4 flow that prints "Hello World". ChatGPT responds with ten steps explaining how to build flow and a copyable code snippet of an entire XML configuration file.
ChatGPT's Default (GPT-3.5) - Build a Mule flow prompt
Code
<?xml version="1.0" encoding="UTF-8"?>

<mule xmlns:ee="http://www.mulesoft.org/schema/mule/ee/core"
      xmlns:http="http://www.mulesoft.org/schema/mule/http"
      xmlns="http://www.mulesoft.org/schema/mule/core"
      xmlns:doc="http://www.mulesoft.org/schema/mule/documentation"
      xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
      xsi:schemaLocation="
        http://www.mulesoft.org/schema/mule/core http://www.mulesoft.org/schema/mule/core/current/mule.xsd
        http://www.mulesoft.org/schema/mule/http http://www.mulesoft.org/schema/mule/http/current/mule-http.xsd
        http://www.mulesoft.org/schema/mule/ee/core http://www.mulesoft.org/schema/mule/ee/core/current/mule-ee.xsd">
    
    <http:listener-config name="HTTP_Listener_config" host="localhost" port="8081" doc:name="HTTP Listener config"/>
    
    <flow name="hello-world">
        <http:listener path="/" config-ref="HTTP_Listener_config" doc:name="HTTP"/>
        <set-payload value="Hello World" doc:name="Set Payload"/>
        <logger level="INFO" doc:name="Logger" message="#[payload]"/>
    </flow>    
</mule>
Observations

Notice that the default version can give you steps to accomplish your task. The output is different from the legacy version. However, it’s interesting that the code uses a “Set-Payload” component over a “Transform Message” component. What could this mean? Could it be that the model still learning DataWeave?


Also, this code provides an entire XML file, instead of a code snippet, unlike the legacy model. And lastly, the code snippet no longer says php, unlike the legacy model that labeled the code as PHP during my February 21st experiment.


GPT-4

In this graphic, I ask ChatGPT's GPT-4 model to Build a Mule 4 flow that prints "Hello World". ChatGPT responds with seven detailed steps with substeps to explain how to build the flow. No code snippet is provided in this response.
ChatGPT's GPT-4 - Build a Mule flow prompt
Now this gets interesting. GPT 4 provides steps like the optimized version of GPT 3.5. But wait? Where is the code?
In this graphic, I ask ChatGPT's GPT4 model to "Provide me with the code. ChatGPT responds with the code snippet and further explanation on how to test the code.
ChatGPT's GPT-4 - Provide code prompt


Code
<?xml version="1.0" encoding="UTF-8"?>

<mule xmlns="http://www.mulesoft.org/schema/mule/core"
    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xmlns:http="http://www.mulesoft.org/schema/mule/http"
    xsi:schemaLocation="
        http://www.mulesoft.org/schema/mule/core http://www.mulesoft.org/schema/mule/core/current/mule.xsd
        http://www.mulesoft.org/schema/mule/http http://www.mulesoft.org/schema/mule/http/current/mule-http.xsd">

    <http:listener-config name="HTTP_Listener_config" doc:name="HTTP Listener config">
        <http:listener-connection host="0.0.0.0" port="8081" />
    </http:listener-config>

    <flow name="HelloWorldFlow" >
        <http:listener doc:name="Listener" config-ref="HTTP_Listener_config" path="/hello"/>
        <set-payload value="Hello World" doc:name="Set Payload" />
        <logger level="INFO" doc:name="Logger" message="#[payload]" />
    </flow>
</mule>
Observations

The first prompt did not provide any code, so I had to give a second prompt. I told ChatGPT to provide me with the code. This took two steps. To prevent having to ask for a code snippet in a subsequent step explicitly, I could have combined the two prompts by explicitly asking ChatGPT to include the code in its response.


Prompt 2: Generate an MUnit test case for this code


Default (GPT 3.5) - Optimized

In this graphic, I ask Default (3.5) - optimized model to Generate an MUnit test case for this code. ChatGPT responds by providing a copyable code snippet of the entire XML file and a description of how to run the MUnit Test case.
ChatGPT's Default (GPT3.5) - Generate a MUnit test case

Code

<?xml version="1.0" encoding="UTF-8"?>

<mule xmlns:ee="http://www.mulesoft.org/schema/mule/ee/core"
      xmlns:http="http://www.mulesoft.org/schema/mule/http"
      xmlns="http://www.mulesoft.org/schema/mule/core"
      xmlns:mock="http://www.mulesoft.org/schema/mule/mock"
      xmlns:munit="http://www.mulesoft.org/schema/mule/munit"
      xmlns:doc="http://www.mulesoft.org/schema/mule/documentation"
      xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
      xsi:schemaLocation="
        http://www.mulesoft.org/schema/mule/core http://www.mulesoft.org/schema/mule/core/current/mule.xsd
        http://www.mulesoft.org/schema/mule/http http://www.mulesoft.org/schema/mule/http/current/mule-http.xsd
        http://www.mulesoft.org/schema/mule/mock http://www.mulesoft.org/schema/mule/mock/current/mule-mock.xsd
        http://www.mulesoft.org/schema/mule/munit http://www.mulesoft.org/schema/mule/munit/current/mule-munit.xsd
        http://www.mulesoft.org/schema/mule/ee/core http://www.mulesoft.org/schema/mule/ee/core/current/mule-ee.xsd">
    
    <http:listener-config name="HTTP_Listener_config" host="localhost" port="8081" doc:name="HTTP Listener config"/>
    
    <flow name="hello-world">
        <http:listener path="/" config-ref="HTTP_Listener_config" doc:name="HTTP"/>
        <set-payload value="Hello World" doc:name="Set Payload"/>
        <logger level="INFO" doc:name="Logger" message="#[payload]"/>
    </flow>
    
    <munit:test name="hello-world-test" description="Test the hello-world flow">
        <mock:when messageProcessor="http:listener" doc:name="Mock HTTP Listener">
            <mock:then-return payload="Hello World"/>
        </mock:when>
        <set-payload value="Hello World" doc:name="Set Payload"/>
        <logger level="INFO" doc:name="Logger" message="#[payload]"/>
        <munit:assert-true message="The payload should be 'Hello World'">
            <munit:expression>#['Hello World' as String = payload as String]</munit:expression>
        </munit:assert-true>
    </munit:test>
</mule>
Observations

At first glance, GPT 3.5 (Default) - optimized seems to understand how to create a general MUnit test case for the first prompt. However, this expression is incorrect.


<munit:expression>#['Hello World' as String = payload as String]</munit:expression>

Can you spot the bug? If so, comment below.


GPT-4


In this graphic, I ask ChatGPT's GPT4  model to Generate an MUnit test case for this code. ChatGPT responds by providing a copyable code snippet of the entire XML configuration file.
ChatGPT's GPT-4 - Generate a MUnit test case

In this graphic, ChatGPT's GPT-4 provides a description of how to run the MUnit Test case provided in the previous graphic.
ChatGPT's GPT-4 - How to run previously generated MUnit test

Code
<?xml version="1.0" encoding="UTF-8"?>
<mule xmlns="http://www.mulesoft.org/schema/mule/core"
    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xmlns:munit="http://www.mulesoft.org/schema/mule/munit"
    xmlns:munit-tools="http://www.mulesoft.org/schema/mule/munit-tools"
    xmlns:http="http://www.mulesoft.org/schema/mule/http"
    xsi:schemaLocation="
        http://www.mulesoft.org/schema/mule/core http://www.mulesoft.org/schema/mule/core/current/mule.xsd
        http://www.mulesoft.org/schema/mule/munit http://www.mulesoft.org/schema/mule/munit/current/mule-munit.xsd
        http://www.mulesoft.org/schema/mule/munit-tools http://www.mulesoft.org/schema/mule/munit-tools/current/mule-munit-tools.xsd
        http://www.mulesoft.org/schema/mule/http http://www.mulesoft.org/schema/mule/http/current/mule-http.xsd">

    <munit:config name="hello-world-flow-test" />
    
    <http:request-config name="HTTP_Request_Configuration" >
        <http:request-connection host="localhost" port="8081" />
    </http:request-config>

    <munit:test name="helloWorldFlowTest" description="Test Hello World flow">
        <munit:execution>
            <http:request method="GET" path="/hello" config-ref="HTTP_Request_Configuration" />
        </munit:execution>

        <munit:validation>
            <munit-tools:assert-that expression="#[payload]" is='#[MunitTools::equalTo("Hello World")]' />
            <munit-tools:assert-that expression="#[attributes.statusCode]" is='#[MunitTools::equalTo(200)]' />
        </munit:validation>
    </munit:test>
</mule>
Observations

GPT 4 provided a slightly different code. For instance, GPT 4 chose not to use a mock when event processor, unlike Default (GPT3.5) - optimized. Instead, it used assert-that event processors.


Another observation was GPT-4 was able to separate the MUnit code by execution and validation. Default (GPT3.5) - optimized did not.


Prompt Engineering

After watching loads of YouTube videos and playing with different generative AI applications, I have found that there is a science to getting the best response. Users of these models must make good prompts. This is so important that a new engineering discipline is forming. It's called prompt engineering. :) Once I get my hands on the next release, I plan to use prompt engineering techniques to improve the responses further.


Additional Comments

Overall, I see a significant improvement from Legacy GPT-3.5 to the Default (GPT-3.5) - optimized and GPT- 4 LLMs. Here are a few.

  1. As promised by OpenAI, both GPT versions received a significant boost to speed.

  2. Both models now produce detailed steps before providing code. This will help you to understand the model's problem-solving and reasoning process.

  3. The detailed steps also show the two model's reasoning has improved. I want to ask the models in my next experiment why they chose specific responses.

  4. Should MuleSoft developers fear ChatGPT? No. However, with Einstein GPT, it would not surprise me if a GPT version is released in an Anypoint Platform product. If that is the case, you should continue learning Mule 4 and DataWeave so you can provide good prompts to a co-pilot tool if such a tool should be released.

Did you like this post? If so, please consider joining my site by filling out the subscription form in the footer and leaving a like or comment below. You will be the first to know when I release part 2 of this post next week.

Recent Posts

See All

Comments


bottom of page