Releases: localstack/localstack
v3.4.0
Summary
This release includes AWS feature updates such as Lambda support for Ruby 3.3 and S3 pre-signed POST policy validation. Enhancements comprise a new experimental event rule engine for event pattern matching and support for Transcribe on Apple Silicon. New resource browsers have been added for LocalStack.
AWS Features
- Lambda now supports the new Lambda runtime Ruby 3.3.
- Support for specific policy validation for pre-signed POST on S3. For S3 pre-signed POST, you can set some conditions when generating the request that the client using the pre-signed POST will need to comply with.
- Support for Redshift Driver in Glue Jobs. (馃専 pro)
- ECS task logs are now streamed to CloudWatch. (馃専 pro)
- EventBridge Pipes now supports EventBridge and StepFunctions as a target. (馃専 pro)
Enhancements
-
Support for a new experimental event rule engine for event pattern matching used in the following services:
- EventBridge
- EventBridge Pipes (馃専 pro)
- Lambda Event Source Mapping
Configure the
EVENT_RULE_ENGINE=java
to use the AWS event-ruler that offers better parity. -
Support for Transcribe on Apple Silicon/
aarch64
. -
While starting an ECS Service, LocalStack now spins up the correct number of tasks. (馃専 pro)
-
Support for the following API methods in AppSync provider: (馃専 pro)
- VTL:
evaluate_mapping_template
- JavaScript:
evaluate_code
- VTL:
-
New Kafka version 3.6.1 is now supported. (馃専 pro)
-
Support for the following APIs are available in the IoT provider: (馃専 pro)
-
Support for RDS Data
BatchExecuteStatement
API. (馃専 pro) -
LocalStack now labels spawned containers with the service name to make identification of running containers easier. (馃専 pro)
-
Support for adding
userName
to user pool Lambda trigger parameters in the Cognito provider. (馃専 pro) -
RDS AuroraMySQL can now be created using the Aurora Versioning Syntax. (馃専 pro)
-
Support for the Step Functions
TestState
API action with the following inspection levels:INFO
DEBUG
TRACE
-
Support for Step Functions
MaxItem
andMaxItemPath
declarations. -
Support for
MaxConcurrencyPath
in Step Functions. -
CloudFormation now supports the following operations:
UPDATE
forAWS::SNS::Subscription
UPDATE
forAWS::SQS::QueuePolicy
CREATE
&DELETE
forAWS::Events::ApiDestination
-
Lambda added a metadata API endpoint
/_aws/lambda/runtimes
to query the supported Lambda runtimes.
LocalStack Features
- New Resource Browsers for:
- Neptune
- Route53 Resolvers
- Cost Explorer
- Account Management
Deprecations
-
The following commands have been removed from the
localstack
CLI experience:localstack login
localstack logout
They have been replaced by
localstack auth login
andlocalstack auth logout
commands.
What's Changed
Exciting New Features 馃帀
- add update operations for AWS::SNS::Subscription and AWS::SQS::QueuePolicy. by @pinzon in #10545
- extend pinning in ASF update action by @alexrashed in #10537
- add script for generating Kubernetes dev files by @pinzon in #10560
- Refactor the TCP proxy into a clearer, server based construct by @dfangl in #10490
- fix lambda function URLs when they don't exist or the service is not loaded by @thrau in #10640
- [SFN] Support for TestState Api Action by @MEPalma in #10168
- add cdk infra setup for ecr images by @pinzon in #10678
- Add experimental event ruler by @joe4dev in #10615
- [SFN] Fix Heartbeat Callback Locking by @MEPalma in #10663
- Add Ruby 3.3 Lambda runtime by @joe4dev in #10704
- Feature: Eventbridge v2: CRUD by @maxhoheiser in #10613
- Add custom Lambda runtimes endpoint by @joe4dev in #10710
Other Changes
- Fix non-default-region compatibility for recent lambda invalid invoke test by @dfangl in #10566
- raise exception if required ENV is missing by @steffyP in #10568
- fix clouformation ec2 tests for
ap-northeast-1
and validate against AWS by @sannya-singal in #10563 - update README after 3.3.0 release by @alexrashed in #10570
- fix SNS RawMessageDelivery casing by @bentsku in #10575
- fix APIGW http integration connection to lambda url by @cloutierMat in #10561
- remove the conditional provider in
test_dashboard_lifecycle
by @sannya-singal in #10571 - Feature: Eventbridge v2: Scaffold new provider by @maxhoheiser in #10552
- fix cfn templates deployed in invalid AZs by @sannya-singal in #10586
- fix lastRotatedDate updation on secret rotation by @macnev2013 in #10564
- update pre-commit hook versions on dep updates by @alexrashed in #10587
- Feat: Events v2: add ci step for eventbridge v2 provider by @maxhoheiser in #10553
- Fix apigw http proxy response passthrough by @cloutierMat in #10583
- [SFN] Fix Evaluation of Nested Map States by @MEPalma in #10574
- Fix release helper for cli release by @silv-io in #10597
- Bump moto-ext to 5.0.4.post1 by @viren-nadkarni in #10589
- [SFN] Makefile Targets for Parsers Generation by @MEPalma in #10193
- fix function get azs cloudformation template by @sannya-singal in #10595
- fix DDB issue when TransactWriteItems targets tables with and without streams by @bentsku in #10593
- chore: remove repetitive words by @veryyet in #10507
- fix SNS MessageBody filtering when value is a list by @bentsku in #10594
- added aws validate test case for ListSecrets filtering by @macnev2013 in #10520
- fix s3 create bucket constraint by @bentsku in #10604
- Revert "added aws validate test case for ListSecrets filtering" by @macnev2013 in #10607
- Add event matching test suite by @joe4dev in #10599
- eventbridge: fix handling of list elements by @baermat in #10600
- fixed tags operations in secretsmanager by @macnev2013 in #10579
- Remove hardcoded credentials and region for unit tests by @viren-nadkarni in #10253
- fix deprecated secret versions preservation in secretsmanager by @macnev2013 in #10572
- added aws validate test case for ListSecrets filtering by @macnev2013 in #10608
- Raise correct exceptions for simultaneous lambda function updates by @dfangl in #10614
- Add container labels to container configuration and container run methods by @dfangl in #10624
- fix secretsmanager test by @alexrashed in #10627
- Add missing library for transcribe on ARM by @silv-io in #10629
- Ensure container clients return the same format for container labels on list call by @dfangl in #10630
- add s3 POST policy validation by @bentsku in #10616
- fix docker client ulimits by @cloutierMat in #10633
- [SFN] Fix Unknown Service sfn errors by @MEPalma in #10631
- fix
connect
unit tests for cross-accounts by @sannya-singal in #10637 - Add update template_body by @lakkeger in #10556
- fix metrics upload failure on community PRs by @alexrashed in #10643
- Check for VPC existence for hostedzone by @simonrw in https://github.com/loc...
v3.3.0
Summary
LocalStack 3.3 is here with a host of new features and enhancements. This release includes support for running ECS tasks on Kubernetes, improved performance for DynamoDB Streams and Kinesis, and a new experimental web server implementation for high-throughput scenarios.
AWS Features
- ECS tasks now can be run when LocalStack is deployed on Kubernetes. To enable this feature, set
ECS_TASK_EXECUTOR=kubernetes
. Tasks can be added to ELB load balancer target groups. (馃専 enterprise) - Lambda now supports the new Dotnet8 runtime.
- Multi-label support is now available for Neptune service. (馃専 pro)
- Filters for list operations and parameters for discover operations is now available in ECS service discovery. (馃専 pro)
- GraphQL APIs created locally by CloudFormation can now be created with tags. (馃専 pro)
- Support for routing of CloudFront aliases to the correct CloudFront distribution is now available. (馃専 pro)
- Support for wildcards in CORS config (eg:
http://*.example.com
) is now available in the S3 service. - Support for ECS Optimised Service Integration, state size quota limits,
MaxItem
,MaxItemPath
, handling ofStates.DataLimitExceeded
errors, and StepFunctions Activities in the Step Functions service. - Support for using external SAML providers (e.g., Auth0) as an identity pool IdP in the Cognito service. (馃専 pro)
Enhancements
- LocalStack now stops and removes underlying EC2 containers while shutting down. You can opt to skip the removal by setting the
EC2_REMOVE_CONTAINERS
flag. (馃専 pro) - IoT Device Shadow Service now supports
DeleteThingShadow
API. (馃専 pro) - MWAA Airflow runners now come with Java Corretto pre-installed. (馃専 pro)
- ARM-based Lambda functions now feature improved test coverage covering all AWS-supported Lambda runtimes.
- Significant performance improvements when DynamoDB Streams or Kinesis destinations are configured for a table (over 15x more throughput for
BatchWriteItem
API). - Significant performance improvements when sending large
INSERT
orCOPY
queries for PostgreSQL database in the RDS service. (馃専 pro) - Default endpoint returned by ECR has been changed to include the Account ID and region for better parity. This can be configured using the new configuration variable
ECR_ENDPOINT_STRATEGY
. (馃専 pro) - Several enhancements for the Step Functions service, including API Actions validation, Map states, JsonPath extraction, and much more!
LocalStack Features
- Support for a new experimental web server implementation that has improved performance for high-throughput scenarios. You can activate it using
GATEWAY_SERVER=twisted
. - New Resource Browser for the CodeCommit service. (馃専 pro)
Deprecations
-
Deprecated six Lambda runtimes including :
python3.7
nodejs14.x
ruby2.7
provided
go1.x
java8
LocalStack prints a deprecation warning but you can still use deprecated runtimes
What's Changed
Exciting New Features 馃帀
- Remove legacy CFn models by @dominikschubert in #10304
- fix functhread daemon status and tmp_thread cleanup by @thrau in #10404
- add twisted gateway server by @thrau in #9834
- implement SQS DLQ redrive to original source queue by @thrau in #10414
- remove contrib package by @thrau in #10424
- make SQS dynamic endpoint strategy guess the endpoint strategy used by @thrau in #10416
- fix SQS DLQ message attributes and redrive into multiple source queues by @thrau in #10425
- use rolo twisted gateway integration by @thrau in #10428
- Support XML responses in api gateway templating by @shucoshuco in #10381
- Feat/firehose add redshift destination by @maxhoheiser in #10375
- KMS: Fixup ImportKeyMaterial for non-symmetric keys by @uubk in #10116
- Update lambda runtime deprecations and fix Lambda ARM builds by @joe4dev in #10441
- Support custom key material when creating KMS keys by @jalaziz in #10379
- add resource provider for Iam Server Certificate by @pinzon in #10455
- Move version single source outside of
__init__.py
by @silv-io in #10497
Other Changes
- fix typo in FAQ link description, closes #10343 by @AnthonyMazzie in #10344
- Fix attributes, create and delete for AWS::Redshift::Cluster by @dominikschubert in #10341
- Fix CircleCI workflow tracking by @silv-io in #10346
- Update readme for v3.2.0 release by @viren-nadkarni in #10228
- fix S3 copy-source format validation by @bentsku in #10338
- add support for Array type in resource provider Scaffolding by @pinzon in #10326
- Fix UserData regression from provider migration in AWS::EC2::Instance by @dominikschubert in #10355
- Added update for "AWS::EC2::Instance" CFn Resource Provider by @Morijarti in #10352
- add timestream to the list of composite apis by @giograno in #10361
- Update kinesis-mock to 0.4.6 by @etspaceman in #10363
- fix workflow execution for PRs coming from forks by @alexrashed in #10366
- [SFN] Base Support for ECS Optimised Service Integration by @MEPalma in #10321
- Added CFn Resource Provider for AWS::SNS::Subscription and AWS::SNS::TopicPolicy by @Morijarti in #10359
- Add remaining migrations that we missed in the initial check by @dominikschubert in #10368
- Improve delete reliability for kinesis streams by @dominikschubert in #10371
- [SFN] Enhancements for Map State by @MEPalma in #10204
- Fix parsing of StepFunction timestamps by @dominikschubert in #9813
- add rolo to dev script by @bentsku in #10384
- Bump moto-ext to 5.0.2.post2 by @viren-nadkarni in #10389
- Disable legacy CFn model execution by @dominikschubert in #10353
- Enhance parity for API Gateway deletion of wildcard method settings by @whummer in #10399
- refactor s3 storage to use context manager to avoid race condition by @bentsku in #10387
- Update CODEOWNERS by @localstack-bot in #10401
- Remove unused CFN_RESOURCE_PROVIDER_OVERRIDES option by @dominikschubert in #10402
- Adding support for partial wildcards for S3 CORS by @cloutierMat in #10365
- add docstring for cw test and assertion by @pinzon in #10394
- fix DDB performance when Streams are enabled by @bentsku in #10415
- update s3 lambdas to node20 by @bentsku in #10417
- fix S3 GLACIER_IR not being an archive type by @bentsku in #10395
- [SFN] Reduce Flakiness of Timeout and Heartbeat Tests by @MEPalma in #10427
- [SFN] Enhancements to api actions validations: InvalidArn, StateMachineDoesNotExist, ExecutionDoesNotExist by @MEPalma in #10421
- fix SNS Publish exception when using empty topic by @bentsku in #10430
- Cfn: Fix bug preventing reuse of stack names by @viren-nadkarni in #10403
- Support new common string functions in velocity templating by @shucoshuco in #10391
- add fix for cfn-response code for nodejs custome resource by @pinzon in #10408
- Move to only using
pyproject.toml
by @silv-io in #10432 - Set package data key to
localstack
by @silv-io in #10444 - Fix DynamoDB Table and Global Table Creation with SSESpecification and TTL by @Morijarti in #10433
- Update runtime init, add dynamic configuration for folders to run chmod on in Lambda by @dfangl in #10438
- Add ARM test selection marker by @joe4dev in #10275
- Introduce test selection for CI tests on PRs by @dominikschubert in #103...
v3.2.0
Summary
LocalStack 3.2.0 release includes new AWS Pinpoint and Textract providers, EventBridge Pipes, and new Resource Browsers for various AWS services.
New Features & Services
- Initial support for a new Pinpoint provider, supporting basic CRUD operations surrounding AWS Pinpoint. (馃専 pro)
- Initial support for a new Textract provider, supporting basic CRUD operations surrounding AWS Textract. (馃専 pro)
- EventBridge now supports Input Transformation. This allows customization of the text from an event before it is passed to the target specified by the respective rule.
Enhancements
- LocalStack is now more intelligent about printing download progress debug messages when the
DEBUG
flag is configured. - OpenSearch now supports v2.11.
- CloudWatch now supports additional regex filter patterns.
- Cognito now supports Pre token generation Lambda
V2_0
trigger event. (馃専 pro) - EventBridge Pipes is being continuously improved. It now features: (馃専 pro)
- Basic poller for DynamoDB streams and SNS sender.
- Improvements to batching and concurrency.
- CloudWatch logging.
- EC2 instance IP address can now be obtained from
DescribeInstances
orRunInstances
API. This will make it easier to set up workflows that involve network access of emulated EC2 instances. (馃専 pro) - EC2 now provides Ubuntu 22.04 and Amazon Linux 2023 AMIs by default. (馃専 pro)
- MWAA now supports Airflow 2.6.3 and 2.7.2 and comes with improved ARM64 compatibility. (馃専 pro)
- Significant improvements to multi-accounts and multi-region setup compatibility.
- RDS now comes with built-in support for the PostGIS extension. (馃専 pro)
LocalStack Features
- New Resource Browsers for the following AWS services are available on LocalStack Web Application:
- ACM
- Amplify
- AppConfig
- Cognito Identity
- DocumentDB
- EKS
- MQ
- MWAA
- QLDB
- Transcribe
- SNS Resource Browser now allows you to publish messages from the LocalStack Web Application.
Deprecations
- MWAA: Airflow versions v1.10.12, v2.0.2, v2.2.2 have reached end-of-support and will be removed in the next major release of LocalStack
- EC2: LocalStack will no longer provide the Ubuntu 20.04 Docker AMI by default from the next major release.
- RDS: Postgres 10 is deprecated and will not be available from the next major release.
- Cloud Pods: Due to a critical security update, some Cloud Pods may break with this release. It is advisable to re-create the Cloud Pods with the latest version of LocalStack.
What's Changed
Exciting New Features 馃帀
- Bump moto-ext to 5.0.0.post1 by @alexrashed in #10112
- Add AWS::EC2::KeyPair by @pinzon in #10100
- Fix: Firehose: Multiple firehose delivery streams not receiving messages from kinesis event stream by @maxhoheiser in #10155
- pin botocore with ASF API updates, add ASF API forward compatibility by @alexrashed in #10183
- remove pins in setup.cfg, add .python-version by @alexrashed in #10195
- Feature: Events: Add input transformers by @maxhoheiser in #10159
- Refactor: Kinesis: Clean-up test fixtures by @maxhoheiser in #10058
- enable Fn::Transform to use CFn macros by @pinzon in #9776
- add opensearch 2.11 by @HarshCasper in #9935
- Change AWS_EXECUTION_ENV set in the lambda init binary by @dfangl in #10212
- CFn: improve error reporting by @simonrw in #10255
- upgrade plux to 1.7 by @thrau in #10266
- update S3 pre-signed credentials logic by @bentsku in #10265
- HTTP IntegrationSubtype support for Kinesis-PutRecord by @calvernaz in #10288
- add support to websockets and sns backend by @calvernaz in #10140
- add option to serve gateway through werkzeug by @thrau in #10171
- Feature: Scenario Tests: Firehose central scenario test by @maxhoheiser in #10243
Other Changes
- Fix prebuilding for lambda functions by @dominikschubert in #9929
- Update README with 3.1.0 version by @simonrw in #10121
- upgrade rolo to 0.3.x by @thrau in #10129
- [SFN] Support for ReverseOrder in GetExecutionHistory Requests by @MEPalma in #10131
- Update CODEOWNERS by @localstack-bot in #10132
- add script to track validations instead of snapshots by @baermat in #10064
- add dynamic SQS endpoint strategy to build SQS queues from called host by @thrau in #10135
- Rework internal lambda routes to avoid router modifications for every spawned environment by @dfangl in #10133
- Enhance implementation of is_comma_delimited_list to fix --services in pods CLI by @whummer in #10142
- add tests for fix to SNS empty filter policy by @bentsku in #10125
- hot fix for ec2 key pair deployment test by @pinzon in #10148
- CFn: improve error message for invalid ref by @simonrw in #10149
- Enhance parity for DynamoDB to Kinesis stream integration by @whummer in #10143
- Fix scenario tests failing with non-default region by @viren-nadkarni in #9149
- fix Kinesis rejection of log level by @bentsku in #10158
- cloudwatch: fix functionality of metrics for multi-accounts and region by @sannya-singal in #9945
- [SFN] SNS Optimised Integration: Automatic Stringification of Message Arguments by @MEPalma in #10165
- fix dynamodb router rule removal by @thrau in #10172
- Fix bootstrap tests failing in non-default region by @viren-nadkarni in #10152
- skip flaky dynamodb streams test by @alexrashed in #10180
- DynamoDB: Fix startup in non-default account ID/region by @viren-nadkarni in #10157
- fix custom status code for lambda URLs by @thrau in #10170
- Update lambda init binary by @dfangl in #10163
- fix disk space issues in GitHub workflow by @alexrashed in #10186
- Fix/multi account kinesis by @macnev2013 in #10188
- [SFN]: Support for CausePath and ErrorPath by @MEPalma in #10130
- add APIGW RequestParametersResolver support for context var by @bentsku in #10176
- stepfunction: fix sns fifo message attributes by @sannya-singal in #10201
- APIGW: add more debug log when lambda proxy response format is wrong by @bentsku in #10206
- Feature: Lambda: Add aws validated test for two lambdas subscribing to same kinensis event stream by @maxhoheiser in #10166
- Stop DNS server on shutdown by @simonrw in #10161
- [SFN] Enhanced Support for Comments Declarations by @MEPalma in #10211
- add automated Python requirements update by @alexrashed in #10219
- fixed multi account support for logs by @macnev2013 in #10202
- Restore DNS settings in container after dns server shuts down by @dfangl in #10221
- Extract snapshot lib by @dominikschubert in #10164
- Add pipes service principal by @joe4dev in #10224
- Simplify Lambda payload parsing by @joe4dev in #10210
- Fix Lambda function alias name checks by @dominikschubert in #10227
- fine-tune workflow triggers by @alexrashed in #10194
- Sqs fix message groups visibility by @baermat in #10223
- Fix lambda perftests for number of functions and versions by @joe4dev in #10209
- Add Lambda tests for large response handling by @dominikschubert in http...
v3.1.0
Summary
LocalStack 3.1 is here with exciting new features and enhancements. It introduces initial support for new providers: EventBridge Pipes, MemoryDB, Account, WAFv2, Identity Store, AMB, ACM PCA, as well as new features for existing services: the Instance Metadata Service (IMDS). Additionally, it offers improvements in DynamoDB, SQS, CloudWatch, RDS, ElastiCache and Lambda. The release also includes enhancements in Cloud Pods, Resource Browsers, and CI Keys Usage inspection.
New Features and Services
- Initial support for a new EventBridge Pipes provider to create Pipes with SQS queues and Kinesis streams as source and target. You can filter events using EventBridge event patterns and enrich events using Lambda. (馃専 pro)
- Initial support for a new MemoryDB provider to create and manage MemoryDB clusters locally, either in the LocalStack container itself or in separate containers for each cluster node. (馃専 pro)
- ElastiCache now supports creating Redis cache nodes and clusters as individual Docker containers, similar to MemoryDB. (馃専 pro)
- Initial support for a new Account provider, supporting basic CRUD operations surrounding AWS account management. (馃専 pro)
- Initial support for a new WAFv2 provider, supporting basic CRUD operations surrounding AWS Web Application Provider. (馃専 pro)
- Initial support for a new ACM PCA provider, supporting basic CRUD operations surrounding AWS Private Certificate Authority. (馃専 pro)
- Initial support for the Instance Metadata Service (IMDS) that exposes an endpoint inside every EC2 instance at
http://169.254.169.254/
. (馃専 pro) - Initial support for the Identity Store provider, supporting basic CRUD operations surrounding AWS IAM Identity Center. (馃専 pro)
- A new LocalStack-native CloudWatch provider is available behind a feature flag:
PROVIDER_OVERRIDE_CLOUDWATCH=v2
, which includes a lot of parity fixes and is designed to be thread safe. - Initial support for a new Managed Blockchain provider, supporting basic CRUD operations surrounding Amazon Managed Blockchain. (馃専 pro)
Enhancements
- DynamoDB now supports Time to Live (TTL) which allows you to define a per-item expiration timestamp that indicates when an item is no longer needed, and delete it automatically.
- SQS now supports dead-letter queue re-drive emulation through
StartMessageMoveTask
,CancelMessageMoveTask
, andListMessageMoveTasks
API operations. - CloudWatch now supports Lambda targets for alarm actions on LocalStack.
- RDS now supports providing admin permission to the master users on RDS for MySQL and MariaDB engines. (馃専 pro)
- Lambda performance improvements for asynchronous invocations along with reduction in
ReadTimeoutError
occurrences during high load on LocalStack.
LocalStack Features
- Cloud Pods
save
andload
commands now show a progress status on the CLI. (馃専 pro) - New Resource Browsers for the following AWS services are available on LocalStack Web Application: OpenSearch, CloudFront, and Application Auto Scaling. In addition, new runtimes have been added to the Lambda Resource Browser. (馃専 pro)
- Users can now inspect CI Keys Usage on the LocalStack Web Application. (馃専 pro)
Deprecations
DNS_LOCAL_NAME_PATTERNS
is removed in favor ofDNS_NAME_PATTERNS_TO_RESOLVE_UPSTREAM
to list domain names that should not be resolved to the LocalStack container, but forwarded to the upstream resolver.
What's Changed
Exciting New Features 馃帀
- Add option to disable AWS_ENDPOINT_URL by @joe4dev in #9792
- Introduce DNS_NAME_PATTERNS_TO_RESOLVE_UPSTREAM config by @simonrw in #9692
- add support for AWS::DynamoDB::GlobalTable resource type by @pinzon in #9790
- Simplify the official docker-compose.yml by @joe4dev in #9822
- Add flag to disable kinesis-mock persistence by @dfangl in #9871
- Transcribe: Add support for transcribing video files by @viren-nadkarni in #9898
- introduce request finalizers into handler chain by @thrau in #9194
- Test lambda performance and mitigate async invoke timeout error (#9917) by @joe4dev in #9917
- add cli option to restart localstack container by @HarshCasper in #9904
- Updates deduplication ID and group ID max length to 128 by @nishadcptn in #9923
- add lambda alarm action for cloudwatch by @steffyP in #9985
- Test lambda performance and mitigate async invoke timeout error v2 by @joe4dev in #9972
- fix lambda create event source mapping duplicates by @thrau in #9996
- fix sqs multi-protocol handling and upgrade botocore by @alexrashed in #9710
- add SNS signing of messages by @bentsku in #9997
- add extension mechanism for exceptions by @thrau in #10019
- Add config option to control maximum gateway worker thread count by @dfangl in #10016
- fix support calling sqs from api ws gw on disconnect endpoint by @calvernaz in #10008
- implement apigw request validation for parameters (header/path/qs) by @bentsku in #10036
- refactor http router for better werkzeug compatibility by @thrau in #10011
- move http framework code to rolo and only keep references by @thrau in #10061
- Make the internal sqs implementation of lambda events the default by @dfangl in #10062
- Add Cloudwatch v2 Provider by @pinzon in #9347
- move gateway and handler chain framework to rolo by @thrau in #10092
Other Changes
- add APIGW PutIntegration IntegrationType validation by @bentsku in #9763
- Enable test for Fn::GetAZs and extend feature by @pinzon in #9740
- fix kms key creation for s3 encryption by @sannya-singal in #9751
- Remove thread-local region, access key and account ID by @viren-nadkarni in #9750
- added util function for time by @macnev2013 in #9736
- Enable AppSync cloudformation resources by @simonrw in #9726
- Add list to dictionary transformation function in provider_utils For CFn by @Morijarti in #9786
- Update Lambda runtime test skip condition by @simonrw in #9760
- Fix alias resource records & add missing id in AWS::Route53::RecordSet by @dominikschubert in #9788
- Add option to capture raw snapshots when running tests by @dominikschubert in #9775
- [SFN] Fix string lexing in Intrinsic Function arguments by @MEPalma in #9783
- Support creation of CloudFormation stacks with the same name after deletion by @dominikschubert in #9748
- Fix StepFunctions execution date formatting by @dominikschubert in #9702
- Fix missing "Pulling container image" message from CLI startup by @simonrw in #9778
- add missing DeadLetterConfig parameter by @baermat in #9773
- fix recursive types when generating TypedDicts by @alexrashed in #9797
- fixed account id for firehose integration by @macnev2013 in #9785
- fix AWS::SQS::Queue update missing QueueArn and QueueUrl by @bentsku in #9744
- Cfn: Fix boto client not using credentials for Lambda PublishVersion by @viren-nadkarni in #9787
- fix: fix failing dashboard lifecycle test for multi-account and region by @sannya-singal in #9799
- [SFN] Enhancements to States I/O, Support for null InputPath and OutputPath by @MEPalma in #9801
- Improve lambda multiruntime tests by @joe4dev in #9798
- Add support for conditions on CFn stack outputs by @dominikschubert in #9814
- add cross-account for CopyObject by @bentsku in #9806
- fix: handle BucketAlreadyOwnedByYou exception when uploading lambda in scenario test cases by @sannya-singal in #9805
- CFn: fix AWS::Events::Rule ref/id/name attributes by @simonrw in #9789
- Bump Postgresql CI dependency by @viren-nadkarni in #9818
- Impleme...
v3.0.2
In this patch release we've fixed a major issue in the localstack
CLI. Since the last release the CLI was missing advanced commands like auth
and pod
, which should now work again.
What's Changed
- fix get_raw_path with double slash prefix path for legacy v-host S3 by @bentsku in #9725
- CFn: render ext providers with correct plugin namespace by @simonrw in #9728
- [SFN] Support for CF Definition fields by @MEPalma in #9677
- [SFN] Enhancements to V2 Interpreter by @MEPalma in #9721
- [SFN] Fix optimised integrations clients leading to timeouts and retries by @MEPalma in #9732
- [SFN] Context Object arguments in Intrinsic Functions by @MEPalma in #9733
- fix cloudwatch resource providers by @pinzon in #9739
- s3: add SourceBucketLocation for CrossLocationLoggingProhibitions exception by @sannya-singal in #9737
- Remove HEADER_LOCALSTACK_ACCOUNT_ID by @viren-nadkarni in #9724
- Remove last uses of get_aws_account_id() and aws_stack.get_region() by @viren-nadkarni in #9641
- Remove localstack client dependency by @simonrw in #9368
- [SFN] Enhancements for Intrinsic Function StringSplit by @MEPalma in #9749
- fix AWS::SNS::Topic canonicalizing bool to str for Attributes by @bentsku in #9743
- create s3 image and pipeline by @bentsku in #9712
- s3: fix test_put_bucket_logging_wrong_target for multi-account and region by @sannya-singal in #9713
- Fix Lambda CloudWatch metric recording upon invocation error by @joe4dev in #9752
- Test lambda provisioned concurrency scheduling by @joe4dev in #9762
- Fix for double-slash uri request by @calvernaz in #9723
- fix type hints incompatible with python < 3.10 in aws_stack by @viren-nadkarni in #9765
- pin global awscli install in Dockerfile by @bentsku in #9767
Full Changelog: v3.0.1...v3.0.2
v3.0.1
What's Changed
- [minor] skip mounting binary pip dependencies in dev container by @whummer in #9585
- unpin opensearch-py by @alexrashed in #9665
- switch to shared pr label enforcers, add bugfix only action by @alexrashed in #9673
- SQS: Rework is_sqs_queue_url() utility by @viren-nadkarni in #9638
- fix s3 skip presign for missing SignedHeaders and fix query string param to headers by @bentsku in #9676
- Fix broken multi-account/multi-region functionality by @viren-nadkarni in #9541
- skip emitting events in SSM if not enabled by @bentsku in #9682
- add cw tests for lambda Invocations and Errors metrics by @steffyP in #9653
- fix get_object_tagging should work on versioned buckets without providing an explicit VersionId by @matt-mercer in #9684
- Rename ARN builders for consistency by @viren-nadkarni in #9618
- Fix execution storage for new stepfunctions provider by @dominikschubert in #9696
- Fix int exception for API GW by @lakkeger in #9672
- Bump moto-ext to 4.2.9.post2 by @viren-nadkarni in #9624
- unpin botocore, update minimum version by @alexrashed in #9667
- Refactor kinesis connector by @dfangl in #9644
- corrects status for instance profile resource provider by @pinzon in #9474
- remove sqs-query from health endpoint by @alexrashed in #9709
- avoid botocore 1.32.6 by @alexrashed in #9714
- apigateway: add IAM role to apigateway integration with services by @sannya-singal in #9639
- fix transfer.ListTagsForResource collect not implemented call by @dfangl in #9716
- Add new lambda runtimes and more parity fixes by @joe4dev in #9697
- add resource providers for EC2::TransitGateway and EC2::TransitGatewayAttachment by @pinzon in #9424
- add AWS::CloudFormation::Stack resource provider by @pinzon in #9449
- fix: support for non-homogeneous lists by @macnev2013 in #9717
- fix SNS publish across region exception by @bentsku in #9679
- fix: supported multi account for test kinesis forward chain by @macnev2013 in #9662
New Contributors
Full Changelog: v3.0.0...v3.0.1
v3.0.0
Summary
This is our third major release for LocalStack featuring both updates to our core services as well as auxiliary tooling. With our third major release for LocalStack we鈥檙e introducing improved service providers for ElastiCache, StepFunctions and S3, improved write performance in DynamoDB, increased Multi-Account and Multi-Region support and much more! A simplified networking configuration, new default container name and lots of internal refactoring to unify endpoint generation will help developers when testing their cloud applications locally.
Additionally to our core emulation in LocalStack we鈥檝e released many additional features available in our Web Application such as IAM policy streams and a Chaos Engineering section to help you bulletproof your applications, both in terms of minimal permissions and increased robustness against failures. Our new Desktop application, LocalStack Desktop, has been released as the successor to our legacy Cockpit App and can be downloaded in the Web Application.
Several of these changes require a migration and we have done our best to make the migration for you as smooth as possible. Head to the How to migrate section and the linked resources there, to find out more. If you have trouble migrating to LocalStack 3.0 please reach out to us, we鈥檙e happy to help!
AWS Features
- Native
v2
provider for StepFunctions (default since 3.0.0) - Native
v3
provider for S3 (default since 3.0.0) - Pro New ElastiCache provider (default since 3.0.0)
- Huge performance improvements for DynamoDB write operations (1.6x for single
PutItem
and up to 10x forBatchWriteItem
) - Improved Multi-Account and Multi-Region support in multiple services such as CloudWatch, StepFunctions, EventBridge, Glue and more.
- The way ARNs are constructed internally has been significantly revamped. This change is particularly beneficial for users working with LocalStack in scenarios involving non-default account IDs or regions.
- Updating Lambda Event Source Mappings to point to different function versions has been fixed to now correctly switch the target and trigger the new version.
- We now offer both a Query and JSON-protocol support for SQS to support recent low-level changes in the SQS API
- DocumentDB has received support for
MasterUsername
andMasterUserPassword
. With the feature flagDOCDB_PROXY_CONTAINER=1
a proxied container is started, and Lambda can connect to the container usingLOCALSTACK_HOSTNAME
- RDS now actually uses MySQL (instead of MariaDB) if the engine聽
mysql
聽is used. It now also correctly returns only the hostname in the聽Endpoint
聽field of responses to the聽CreateDBCluster
聽operation. - Lots of incremental improvements to CloudFormation
- Improved detection of undefined dependencies in templates
- Support for Fn::Select in conditionals
- Fix AWS::S3::Bucket EventBridge Notifications
- Resources in Community
- (New) AWS::EC2::NetworkAcl
- (New) AWS::EC2::DHCPOptions
- (Updated) AWS::DynamoDB::Table
- (Updated) AWS::OpenSearch::Domain
- Resources in Pro
- (New) AWS::ElasticBeanstalk::Application
- (New) AWS::ElasticBeanstalk::ApplicationVersion
- (New) AWS::ElasticBeanstalk::Environment
- (New) AWS::ElasticBeanstalk::ConfigurationTemplate
- (Updated) AWS::RDS::DBCluster
LocalStack Features
- We now create native M1 binaries of our CLI. When installing via Brew these are automatically selected for you depending on your CPU architecture.
- Since 3.0 we publish major tags. You can pin the current major version like this:
localstack/localstack-pro:3
to stay on the latest tagged 3.x release but avoid unintentionally upgrading to the next major version. - With 3.0 we鈥檙e also now for the first time publishing a
stable
tag, which will always point towards the latest tagged release. - The localstack docker image is now based on Python 3.11 and Debian Bookworm
- The default container name for LocalStack is now
localstack-main
(previouslylocalstack_main
) to allow the use of the name in URLs. - Networking configuration has been simplified by removing legacy options and moving to
GATEWAY_LISTEN
andLOCALSTACK_HOST
. Check out our Discuss page for more information. - Strict Service Loading allows you to define a strict subset of services to load via the
SERVICES
variable, e.g.SERVICES=s3
. Any services not listed here will now be prevented from loading. - Lots of enhancements for LocalStack Extensions
- LocalStack Desktop is replacing the deprecated Cockpit Desktop UI
- The IAM policy stream is a new feature in our Web Application that allows you to see exactly which permissions are needed for the API calls you鈥檙e making in LocalStack.
- We鈥檝e improved the Chaos Engineering capabilities of LocalStack, extending or FIS (Fault Injection Simulator) implementation and added a UI to interface with it in our Web Application.
Provider Deprecations
- StepFunctions
legacy
/v1
provider has been deprecated and will be removed in the next major version. - S3
v2
provider has been deprecated and will be removed in the next major version. - ElastiCache
legacy
provider has been deprecated and will be removed in the next major version.
Removals
- Lambda
legacy
/v1
provider has been removed. - S3
legacy
/v1
provider has been removed. - We鈥檝e removed a lot of legacy code with this major release. If you鈥檝e been using any internal constructs from localstack, e.g. via extensions, please make sure they are still working. Feel free to reach out to us if you need help migrating your extension.
- Community Cloud Pods have been removed.
- After its initial preview phase, AWS RAM is now part of our paid offering and will be further developed and integrated with our IAM enforcement in future releases.
- Configuration
DEFAULT_REGION
,USE_SINGLE_REGION
ES_ENDPOINT_STRATEGY
,聽ES_MULTI_CLUSTER
,聽ES_CUSTOM_BACKEND
KINESIS_INITIALIZE_STREAMS
SYNCHRONOUS_*_EVENTS
EC2_AUTOSTART_DAEMON
*_PORT_EXTERNAL
- legacy
LAMBDA_*
settings, see the sections below for more details
How to migrate
Networking
We have published an extensive migration guide for our networking related changes on this Discuss page
- The default container name is renamed from
localstack_main
tolocalstack-main
- Our default container name
localstack_main
is not a valid URL, so when the container name is used as a hostname, some libraries refuse to connect. With this release, we use the default container name oflocalstack-main
. There may be breakages with tooling that expects the new container name, but a container is running with the old container name. Please update your tooling accordingly. - Most prominently, this is container name used in
docker-compose
files. - If you are using the LocalStack CLI, please just make sure to update to the latest version after the release. The CLI will handle this migration automatically.
- Our default container name
HOSTNAME_EXTERNAL
andLOCALSTACK_HOSTNAME
are removed in favor ofLOCALSTACK_HOST
- We are simplifying how to configure LocalStack by fully switching over to use
LOCALSTACK_HOST
to configure URLs returned by AWS services. Please update your usages ofHOSTNAME_EXTERNAL
(default:localhost
) andLOCALSTACK_HOSTNAME
to useLOCALSTACK_HOST
(default:localhost.localstack.cloud:4566
). You can find more details in our Networking Migration Guide for LocalStack 3.0.
- We are simplifying how to configure LocalStack by fully switching over to use
New default protocol for the SQS API
AWS recently updated the default protocol for the SQS API in many of their SDKs, which started breaking with LocalStack since we didn't have JSON support yet. With 3.0.0 we now support both protocols, but if you are still on 2.x a workaround would be to downgrade or pin your used SDK/tool version.
Here is the list with all the SDK versions where the issue will start appearing:
Language | SDK client repository | Required SDK client version |
---|---|---|
C++ | aws/aws-sdk-cpp | 1.11.98 |
Golang 1.x | aws/aws-sdk-go | v1.47.7 |
Golang 2.x | aws/aws-sdk-go-v2 | v1.28.0 |
Java 1.x | aws/aws-sdk-java | 1.12.585 |
Java 2.x | aws/aws-sdk-java-v2 | 2.21.19 |
JavaScript v2.x | aws/aws-sdk-js | v2.1492.0 |
JavaScript v3.x | aws/aws-sdk-js-v3 | v3.447.0 |
.NET | aws/aws-sdk-net | 3.7.681.0 |
PHP | [aws/aws-sdk-php](https://github.c... |
v2.3.2
What's Changed
- Implementation of AWS::Kinesis::StreamConsumer for CloudFormation by @Morijarti in #9249
- Ensure unique addresses for GATEWAY_LISTEN by @simonrw in #9263
- fix SNS FilterPolicy with null value by @bentsku in #9264
- exclude werkzeug 3.0 by @thrau in #9272
- Implementation of CloudFormation support for AWS::KinesisFirehose by @Morijarti in #9259
- add label sync, migrate workflows to shared workflows by @alexrashed in #9238
- Add develop mode to dev script by @simonrw in #9274
- Separate resource provider plugins by @simonrw in #9267
- cleanup and refactor scenario tests by @steffyP in #9251
Full Changelog: v2.3.1...v2.3.2
v2.3.1
What's Changed
- fix service router test by excluding bedrock by @alexrashed in #9256
- Ss18/enhancement0 by @ss18 in #9252
- Add Kinesis Stream resource provider for CloudFormation by @Morijarti in #9230
- Update readme for v2.3.0 by @giograno in #9254
New Contributors
Full Changelog: v2.3.0...v2.3.1
v2.3.0
Summary
LocalStack v2.3. is live! We introduced a new set of services, launched a brand new implementation for core services like S3 and StepFunctions, and shipped significant improvements both in our CLI and web app experience.
New Features and Services
- Initial support for Delta Lake in AWS Athena. (馃専 pro)
- Introducing a new service: EventBridge Scheduler with support for all 12 operations.
- Introducing a new service: Resource Access Manager with support for 5 operations.
- A new LocalStack-native S3 provider is available behind a feature flag:
PROVIDER_OVERRIDE_S3=v3
, with no external dependencies. Improved parity, better memory usage, faster, better support for bucket versioning and multipart upload. - A new LocalStack-native StepFunctions provider is available behind a feature flag:
PROVIDER_OVERRIDE_STEPFUNCTIONS=v2
.
Enhancements
- 12 new operations supported for Amazon Elastic File System (EFS). (馃専 pro)
- Multi-accounts are now supported in Glue (馃専 pro), CloudFormation, CloudWatch (馃専 pro), DynamoDB Streams, and Event Bridge.
- Improved AppSync integration with RDS, supporting variable maps and cross-region access. (馃専 pro)
- Several parity improvements for Elastic Load Balancing V2 and RDS. (馃専 pro)
- Significant improvements to CloudFormation.
Scan
andQuery
operations are now supported for Global Tables in DynamoDB.- A new AWS Lambda data plane implementation brings a better experience upon Lambda runtime startup errors and timeouts, and parity improvements for both asynchronous Lambda invocations and concurrency.
- New Lambda runtime Python 3.11.
- Networking improvements:聽
localhost.localstack.cloud
聽now resolves to the LocalStack container within Docker networks. This is automatically set up for Lambda, ECS, and EC2, and documented for your own containers. - A new configuration variable
GATEWAY_LISTEN
to configure the container bind address and ports.
LocalStack Features
- Improved Pro CLI experience: use Pro CLI commands without logging in. (馃専 pro)
- Significant improvements to the
localstack extensions
CLI UX. (馃専 pro) - Our web application now features a fresh new layout with a sidebar, which should provide a better overview of your resources.
- With the 1.0 release, we introduced LocalStack Extensions to allow developers to extend and customize LocalStack.
We now feature 7 extensions in our library, that allow you to integrate other services, such as聽stripe
,聽mailhog
, and many others, into your local development setup, or even into your CI pipeline with the newEXTENSION_AUTO_INSTALL
config!
These extensions are also featured in our Extensions Library on the web application, which makes the installation and management of Extensions as simple as the click of a button.
Furthermore, you can generate LocalStack Extensions badges for your own extensions, which will allow everyone visiting your repository to install your extension through our web UI. - LocalStack Instance Bookmarks allow you to quickly connect to multiple LocalStack instances running on different endpoints or even remote machines. The new instance layout now also brings together all instance-related pages like instance status, the resource browser, extensions, and export/import of instance state.
- We have revamped the UI for Cloud Pods, which now allows you to view a set of useful metadata like the services contained in the pod and the creation time.
Additionally, the UI allows you to manage your pod versions, with a nifty auto-delete feature.
Furthermore, you can now create and browse the state of your pods on a per-version level directly in the UI! - (beta) We introduced alternative remote storage for Cloud Pods. You will be now able to save your LocalStack state into S3 containers and OCI registries.
- We now support Route 53 and SageMaker Inference in our resource browser!
Deprecations
- Dropped support for Python 3.7 after End-Of-Life.
- The legacy Cloud Pods client will be removed with the next major release.
- Dropped support for the
localstack/bigdata
image.
What's Changed
Exciting New Features 馃帀
- Remove usage of GenericBaseModel outside of LegacyResourceProvider by @dominikschubert in #8711
- S3 AWS-chunked stream decoder by @bentsku in #8721
- move legacy S3 provider into legacy directory by @bentsku in #8713
- [SFN] Overhaul and enhancement of Map states, enhancements Parallel states, tests by @MEPalma in #8720
- [SFN] Support for ItemSelector in Map states by @MEPalma in #8723
- [SFN] Support for Map legacy Iterator declarations by @MEPalma in #8742
- 8701 enhancement request custom lambda authorizer apikeysource is not validated by @calvernaz in #8725
- Introducing EventBridge Scheduler by @viren-nadkarni in #8754
- use ContainerConfiguration object for LocalstackContainer by @thrau in #8771
- [SFN] base support for DynamoDb by @MEPalma in #8323
- improve docker integration with localstack start command by @thrau in #8772
- Allow asterisk only wild card by @fatal10110 in #8572
- Fix outdated lambda runtime images in tooling by @joe4dev in #8780
- [SFN] Publish SFN events to EventBridge by @MEPalma in #8778
- upgrade kinesis-mock to scalajs release, add node installer by @alexrashed in #8790
- configure MAIN_DOCKER_NETWORK automatically if --network flag is used by @thrau in #8818
- add pytest-tinybird to all CI runs on master by @alexrashed in #8807
- [SFN] Fix execution input, fix handling lambda payload i/o by @MEPalma in #8821
- [SFN] Parse fail on incorrect derivations, fix mulitple retry, fix language definitions by @MEPalma in #8826
- introduce new S3 native provider by @bentsku in #8786
- implement S3 native multiparts operations by @bentsku in #8787
- Standardize lambda package structure by @joe4dev in #8861
- implement S3 native bucket versioning by @bentsku in #8799
- add S3 native bucket encryption and AWS KMS managed key by @bentsku in #8800
- implement S3 native notifications by @bentsku in #8802
- add WebSocket support for ASGI/WSGI bridge and Gateway by @thrau in #8835
- [SFN] Enhanced support for empty fail states by @MEPalma in #8829
- implement S3 native tagging by @bentsku in #8811
- DDBStreams: Multi-accounts compatibility by @viren-nadkarni in #8879
- implement S3 native CORS by @bentsku in #8814
- implement S3 native Lifecycle + mocked Bucket APIs by @bentsku in #8863
- implement S3 native Website by @bentsku in #8864
- implement S3 native object lock by @bentsku in #8884
- implement S3 native RequestPayment, GetObject Part, Preconditions by @bentsku in #8885
- DynamoDB: Global table support for Scan and Query by @viren-nadkarni in #8905
- Apply response template to sqs int by @calvernaz in #8899
- implement S3 native PublicAccessBlock and ObjectOwnership by @bentsku in #8886
- implement S3 native Bucket Policy and AccelerateConfig by @bentsku in #8889
- implement S3 native BucketLogging and BucketReplication by @bentsku in #8901
- [SFN] Support for UpdateStateMachine by @MEPalma in #8852
- [SFN] Support for sync:2 by @MEPalma in #8900
- [SFN] Support for State Machine versioning by @MEPalma in #8894
- fix close call propagation of response in asgi/wsgi bridge by @thrau in #8926
- implement S3 native ACL by @bentsku in #8902
- [SFN] Skip local iam role assume check by @MEPalma in #8944
- implement S3 native pre-signed POST (PostObject) by @bentsku in #8907
- implement S3 v3 pre-signed handler by @bentsku in #8918
- Cfn: Multi-accounts compatibility by @viren-nadkarni in #8822
- [SFN] Support for ApiGateway by @MEPalma in https://github.com/localstack/loc...