[test] Add support to run yul optimizer assembly test. #15009

aarlt · 2024-04-11T22:33:02Z

This is needed in the context of ethdebug. I will need to write some tests on how optimiser steps are dealing with the debug attributes defined by #14857. The ideas is to implement different PRs per optimiser step on top of #14969, where this PR will enable the visualisation how these debug attributes got potentially merged - depending on the optimiser step.

For context, see e.g. #15087.

cameel

I see this is pretty much an exact copy of YulOptimizerTest.h at this point, so I assume it's unfinished. What is actually the idea behind this test? Just displaying the assembly next to the optimized Yul or something more?

cameel · 2024-04-12T22:19:07Z

test/libyul/YulOptimizerAssemblyTest.cpp

+
+	// Re-parse new code for compilability
+	// TODO: support for wordSizeTransform which needs different input and output dialects
+	if (m_optimizerStep != "wordSizeTransform" && !std::get<0>(parse(_stream, _linePrefix, _formatted, printed)))


wordSizeTransform seems to be some Ewasm left-over. We have no such step now.

I think we can safely remove this from YulOptimizerTest and a few other places that still reference it.

It was marked as resolved, but this code is still here.

test/InteractiveTests.h

test/libyul/YulOptimizerAssemblyTest.cpp

ekpyron · 2024-05-13T14:27:51Z

I see this is pretty much an exact copy of YulOptimizerTest.h at this point, so I assume it's unfinished. What is actually the idea behind this test? Just displaying the assembly next to the optimized Yul or something more?

Just to answer this: the target here is to test the transport of the debugging context through any individual optimizer step and then to assembly (which is from where we'll output them in the end)

In any case, doesn't hurt to have, and not too much of a headache to just merge.

cameel · 2024-05-16T17:11:49Z

test/libyul/YulOptimizerAssemblyTest.cpp

+
+	// Re-parse new code for compilability
+	// TODO: support for wordSizeTransform which needs different input and output dialects
+	if (m_optimizerStep != "wordSizeTransform" && !std::get<0>(parse(_stream, _linePrefix, _formatted, printed)))


It was marked as resolved, but this code is still here.

cameel · 2024-05-16T17:15:49Z

test/libyul/YulOptimizerAssemblyTest.cpp

+	auto dialectName = m_reader.stringSetting("dialect", "evm");
+	m_dialect = &dialect(dialectName, solidity::test::CommonOptions::get().evmVersion());


With Ewasm gone, adding an option for dialect selection in a newly written test case seems pointless to me. Technically we still support the evmTyped and yul dialects, but have you even checked if they work here? That would be a waste of time anyway and IMO would be much better to just remove this option and assume the evm dialect.

You could then also get rid of dialect checks in the rest of the test case.

cameel · 2024-05-16T17:27:58Z

test/libyul/YulOptimizerAssemblyTest.cpp

+	EVMObjectCompiler::compile(
+		*m_object,
+		adapter,
+		EVMDialect::strictAssemblyForEVMObjects(EVMVersion{}),


Why aren't we respecting the dialect and EVM version selected by the user here even though we do in the other places in this test case?

Same question for analyzeStrictAssertCorrect() above.

Also, it would be a good idea to have something EVM version-specific in the sample test case so that we can see if this works correctly.

cameel · 2024-05-16T17:29:56Z

test/libyul/yulOptimizerAssemblyTests/blockFlattener/basic.yul

+    {
+        let _1 := mload(0)
+        let f_a := mload(1)
+        let f_r
+        {
+            f_a := mload(f_a)
+            f_r := add(f_a, calldatasize())
+        }
+        let z := mload(2)
+    }


Well, if the new test is meant for inspecting the debug data, then it would be useful to actually have some debug annotations in the sample test to see if it's even preserved in the output.

cameel · 2024-05-16T17:37:06Z

test/libyul/YulOptimizerAssemblyTest.cpp

+		false,
+		std::nullopt


Suggested change

false,

std::nullopt

false, // _optimize

std::nullopt // _eofVersion

I think it this should be using the value supplied to --eof-version (i.e. solidity::test::CommonOptions::get().eofVersion()). Which will still be nullopt, by default, but I see no reason to ignore this option if supplied.

As for optimization, I guess we don't want because that's only the legacy optimizer? If so, at least a comment saying that would be nice. Otherwise these choices seem arbitrary.

cameel · 2024-05-16T17:38:49Z

test/libyul/YulOptimizerAssemblyTest.cpp

+	std::ostringstream output;
+	output << assembly;
+	m_obtainedResult += "\nAssembly:\n" + output.str();


This pattern is so common that we already have a helper for it in CommonIO.h:

Suggested change

std::ostringstream output;

output << assembly;

m_obtainedResult += "\nAssembly:\n" + output.str();

m_obtainedResult += "\nAssembly:\n" + toString(assembly);

cameel · 2024-05-16T17:58:28Z

test/libyul/YulOptimizerAssemblyTest.cpp

+
+TestCase::TestResult YulOptimizerAssemblyTest::run(std::ostream& _stream, std::string const& _linePrefix, bool const _formatted)
+{
+	std::tie(m_object, m_analysisInfo) = parse(_stream, _linePrefix, _formatted, m_source);


m_object and m_analysisInfo are never used outside of this function. What's the point of even having them as class members rather than locals?

cameel · 2024-05-16T18:07:28Z

test/libyul/YulOptimizerAssemblyTest.cpp

+std::pair<std::shared_ptr<Object>, std::shared_ptr<AsmAnalysisInfo>> YulOptimizerAssemblyTest::parse(
+	std::ostream& _stream,
+	std::string const& _linePrefix,
+	bool const _formatted,
+	std::string const& _source
+)
+{
+	ErrorList errors;
+	soltestAssert(m_dialect, "");
+	std::shared_ptr<Object> object;
+	std::shared_ptr<AsmAnalysisInfo> analysisInfo;
+	std::tie(object, analysisInfo) = yul::test::parse(_source, *m_dialect, errors);
+	if (!object || !analysisInfo || Error::containsErrors(errors))
+	{
+		AnsiColorized(_stream, _formatted, {formatting::BOLD, formatting::RED}) << _linePrefix << "Error parsing source." << std::endl;
+		CharStream charStream(_source, "");
+		SourceReferenceFormatter{_stream, SingletonCharStreamProvider(charStream), true, false}
+			.printErrorInformation(errors);
+		return {};
+	}
+	return {std::move(object), std::move(analysisInfo)};
+}


Looks like it was copied verbatim from YulOptimizerTest. Why not just reuse it? Other than m_dialect (which could be parameter), this could easily be converted into a static function or a standalone helper and just called by this class.

cameel · 2024-05-16T18:25:40Z

test/libyul/YulOptimizerAssemblyTest.cpp

+	m_object->analysisInfo = std::make_shared<AsmAnalysisInfo>(
+		AsmAnalyzer::analyzeStrictAssertCorrect(EVMDialect::strictAssemblyForEVMObjects(solidity::test::CommonOptions::get().evmVersion()), *m_object)
+	);


Is it really correct to simply overwrite the previous analysis info? Do we do that in real compilation too?

cameel mentioned this pull request Apr 12, 2024

Clean up tab-space soup #15026

Merged

cameel reviewed Apr 12, 2024

View reviewed changes

cameel added the testing 🔨 label Apr 12, 2024

aarlt force-pushed the add_yul_optimizer_assembly_test branch from e953eca to 917cec1 Compare April 18, 2024 21:01

aarlt commented Apr 22, 2024

View reviewed changes

test/libyul/YulOptimizerAssemblyTest.cpp Outdated Show resolved Hide resolved

aarlt mentioned this pull request Apr 22, 2024

Remove wordSizeTransform left-over. #15047

Merged

aarlt force-pushed the add_yul_optimizer_assembly_test branch 4 times, most recently from 8a4d556 to de39abd Compare April 22, 2024 15:19

github-actions bot added the stale The issue/PR was marked as stale because it has been open for too long. label May 7, 2024

ethereum deleted a comment from github-actions bot May 7, 2024

aarlt removed the stale The issue/PR was marked as stale because it has been open for too long. label May 7, 2024

[test] Add support to run yul optimizer assembly test.

9357780

aarlt force-pushed the add_yul_optimizer_assembly_test branch from de39abd to 9357780 Compare May 7, 2024 12:19

aarlt marked this pull request as ready for review May 8, 2024 16:07

aarlt requested a review from cameel May 8, 2024 16:08

This was referenced May 9, 2024

[ethdebug] Transport debug data through block flattener. #15087

Draft

[ethdebug] Transport debug data through common subexpression eliminator. #15088

Draft

ekpyron approved these changes May 13, 2024

View reviewed changes

cameel reviewed May 16, 2024

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[test] Add support to run yul optimizer assembly test. #15009

[test] Add support to run yul optimizer assembly test. #15009

aarlt commented Apr 11, 2024 •

edited

cameel left a comment •

edited

cameel Apr 12, 2024

cameel May 16, 2024

ekpyron commented May 13, 2024

cameel May 16, 2024

cameel May 16, 2024 •

edited

cameel May 16, 2024 •

edited

cameel May 16, 2024

cameel May 16, 2024

cameel May 16, 2024

cameel May 16, 2024

cameel May 16, 2024

cameel May 16, 2024 •

edited

		auto dialectName = m_reader.stringSetting("dialect", "evm");
		m_dialect = &dialect(dialectName, solidity::test::CommonOptions::get().evmVersion());

-		false,
-		std::nullopt
+		false,       // _optimize
+		std::nullopt // _eofVersion

[test] Add support to run yul optimizer assembly test. #15009

Are you sure you want to change the base?

[test] Add support to run yul optimizer assembly test. #15009

Conversation

aarlt commented Apr 11, 2024 • edited

cameel left a comment • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

ekpyron commented May 13, 2024

Choose a reason for hiding this comment

cameel May 16, 2024 • edited

Choose a reason for hiding this comment

cameel May 16, 2024 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

cameel May 16, 2024 • edited

Choose a reason for hiding this comment

aarlt commented Apr 11, 2024 •

edited

cameel left a comment •

edited

cameel May 16, 2024 •

edited

cameel May 16, 2024 •

edited

cameel May 16, 2024 •

edited