Merge branch 'release/5.4.0'

* Fixing forced OpCache Invalidation on every call, which is resulting in fast raising wasted memory
* Fix undefined $path variable warning
---------

Co-authored-by: Daniel Metzner <daniel.metzner@niceshops.com>

CHANGELOG.md

@@ -6,6 +6,20 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 
 ## [Unreleased]
 
+## [5.4.0] - 2024-08-14
+- Fixing forced OpCache invalidation on every template include, which is resulting in fast raising wasted OpCache memory [#1007](https://github.com/smarty-php/smarty/issues/1007) 
+- Improvement of auto-escaping [#1030](https://github.com/smarty-php/smarty/pull/1030)
+
+
+## [5.3.1] - 2024-06-16
+- Fixed error when using section with nocache [#1034](https://github.com/smarty-php/smarty/issues/1034)
+
+
+## [5.3.0] - 2024-05-30
+- Fix warning when calling hasVariable for an undefined variable [#977](https://github.com/smarty-php/smarty/issues/977)
+- Added `$smarty->prependTemplateDir()` method [#1022](https://github.com/smarty-php/smarty/issues/1022)
+
+
 ## [5.2.0] - 2024-05-28
 - Fixed a code injection vulnerability in extends-tag. This addresses CVE-2024-35226.
 - Added `$smarty->setCacheModifiedCheck()` setter for cache_modified_check
@@ -22,16 +36,16 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 - Internal compiler classes always return a string (the internal has_code flag has been removed for simplicity) [#918](https://github.com/smarty-php/smarty/pull/918)
 - Fix invalid classnames in Runtime code for foreach [#1000](https://github.com/smarty-php/smarty/issues/1000)
 
+## [5.0.2] - 2024-03-28
+- Fix Smarty::assign() not returning $this when called with an array as first parameter [#972](https://github.com/smarty-php/smarty/pull/972)
 
 ## [5.0.1] - 2024-03-27
 - Fix error in Smarty\Smarty::compileAllTemplates() by including missing FilesystemIterator class [#966](https://github.com/smarty-php/smarty/issues/966)  
 
-
 ## [5.0.0] - 2024-03-25
 - Fixed that scoped variables would overwrite parent scope [#952](https://github.com/smarty-php/smarty/issues/952)
 - Removed publicly accessible `$tpl->_var_stack` variable
 
-
 ### Fixed
 - Too many shorthand attributes error when using a modifier as a function with more than 3 parameters in an expression [#949](https://github.com/smarty-php/smarty/issues/949)
 

changelog/1022.md

@@ -1 +0,0 @@
-- Added `$smarty->prependTemplateDir()` method [#1022](https://github.com/smarty-php/smarty/issues/1022)

changelog/972.md

@@ -1 +0,0 @@
-- Fix Smarty::assign() not returning $this when called with an array as first parameter [#972](https://github.com/smarty-php/smarty/pull/972)

changelog/979.md

@@ -1 +0,0 @@
-- Fixed missing Smarty getErrorUnassigned/setErrorUnassigned methods [#979](https://github.com/smarty-php/smarty/issues/979)

docs/api/configuring.md

@@ -143,6 +143,35 @@ Enable auto-escaping for HTML as follows:
 $smarty->setEscapeHtml(true);
 ```
 
+When auto-escaping is enabled, the `|escape` modifier's default mode (`html`) has no effect,
+to avoid double-escaping. It is possible to force it with the `force` mode.
+Other modes (`htmlall`, `url`, `urlpathinfo`, `quotes`, `javascript`) may be used
+with the result you might expect, without double-escaping.
+
+Even when auto-escaping is enabled, you might want to display the content of a variable without
+escaping it. To do so, use the `|raw` modifier.
+
+Examples (with auto-escaping enabled):
+```smarty
+{* these three statements are identical *}
+{$myVar}
+{$myVar|escape}
+{$myVar|escape:'html'}
+
+{* no double-escaping on these statements *}
+{$var|escape:'htmlall'}
+{$myVar|escape:'url'}
+{$myVar|escape:'urlpathinfo'}
+{$myVar|escape:'quotes'}
+{$myVar|escape:'javascript'}
+
+{* no escaping at all *}
+{$myVar|raw}
+
+{* force double-escaping *}
+{$myVar|escape:'force'}
+```
+
 ## Disabling compile check
 By default, Smarty tests to see if the
 current template has changed since the last time

docs/designers/language-modifiers/language-modifier-escape.md

@@ -73,6 +73,6 @@ This snippet is useful for emails, but see also
 <a href="mailto:{$EmailAddress|escape:'hex'}">{$EmailAddress|escape:'mail'}</a>
 ```
 
-See also [escaping smarty parsing](../language-basic-syntax/language-escaping.md),
+See also [auto-escaping](../../api/configuring.md#enabling-auto-escaping), [escaping smarty parsing](../language-basic-syntax/language-escaping.md),
 [`{mailto}`](../language-custom-functions/language-function-mailto.md) and the [obfuscating email
-addresses](../../appendixes/tips.md#obfuscating-e-mail-addresses) page.
+addresses](../../appendixes/tips.md#obfuscating-e-mail-addresses) pages.

docs/designers/language-modifiers/language-modifier-raw.md

@@ -0,0 +1,8 @@
+# raw
+
+Prevents variable escaping when [auto-escaping](../../api/configuring.md#enabling-auto-escaping) is activated.
+
+## Basic usage
+```smarty
+{$myVar|raw}
+```

docs/img/smarty.svg

@@ -0,0 +1,22 @@
+<?xml version="1.0" encoding="UTF-8"?>
+<!-- Generator: Adobe Illustrator 25.2.1, SVG Export Plug-In . SVG Version: 6.00 Build 0)  -->
+<svg version="1.1" id="Layer_1" xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" x="0px" y="0px" viewBox="0 0 1000 1000" style="enable-background:new 0 0 1000 1000;" xml:space="preserve">
+<style type="text/css">
+	.st0{fill:#F5E0B4;}
+	.st1{fill:#383535;}
+	.st2{fill:#FBDE3D;}
+	.st3{fill:#8C919F;}
+	.st4{fill:#CACED5;}
+	.st5{fill:#FFFFFF;}
+</style>
+<g>
+	<polygon class="st0" points="782.2,925.8 733.4,643.2 996.4,532.7 745.7,396.4 823.6,120.2 560.7,232.2 391.6,0 313.7,274.7    26.5,260.1 194.4,491.3 3.6,705.5 289.4,721.3 340.5,1000 529.1,787  "/>
+	<path class="st1" d="M706.9,493.4c0-126-102.2-228.2-228.2-228.2c-126,0-228.2,102.2-228.2,228.2c0,81.8,43.1,153.6,107.8,193.8   v202.6l43.5,43.5l0.2-0.2h150.6l0.2,0.2l43.5-43.5V689C662.6,649,706.9,576.4,706.9,493.4z"/>
+	<path class="st2" d="M682.2,492.9c0-112.7-91.3-204-204-204s-204,91.3-204,204c0,78,43.8,145.8,108.1,180.1v93.6h191.7V673   C638.4,638.7,682.2,571,682.2,492.9z"/>
+	<g>
+		<polygon class="st3" points="382.7,790.4 382.7,881.1 412,910.3 544.9,910.3 574.1,881.1 574.1,790.4   "/>
+		<polygon class="st4" points="540,813.2 540,866.5 527,879.5 540.4,892.9 558.7,874.6 558.9,874.6 558.9,813.2   "/>
+	</g>
+	<path class="st5" d="M478.3,310.4c-5,0-9.1,4.1-9.1,9.1c0,5,4.1,9.1,9.1,9.1c90.7,0,164.3,73.5,164.3,164.3c0,5,4.1,9.1,9.1,9.1   s9.1-4.1,9.1-9.1C660.8,392.1,579.1,310.4,478.3,310.4z"/>
+</g>
+</svg>

make-release.sh

@@ -15,7 +15,7 @@ php utilities/update-smarty-version-number.php $1
 git add changelog CHANGELOG.md src/Smarty.php
 git commit -m "version bump"
 
-git checkout support/5
+git checkout master
 git pull
 git merge --no-ff "release/$1"
 git branch -d "release/$1"

mkdocs.yml

@@ -68,6 +68,7 @@ nav:
         - 'noprint': 'designers/language-modifiers/language-modifier-noprint.md'
         - 'number_format': 'designers/language-modifiers/language-modifier-number-format.md'
         - 'nl2br': 'designers/language-modifiers/language-modifier-nl2br.md'
+        - 'raw': 'designers/language-modifiers/language-modifier-raw.md'
         - 'regex_replace': 'designers/language-modifiers/language-modifier-regex-replace.md'
         - 'replace': 'designers/language-modifiers/language-modifier-replace.md'
         - 'round': 'designers/language-modifiers/language-modifier-round.md'

src/Compile/Modifier/EscapeModifierCompiler.php

@@ -24,22 +24,32 @@ class EscapeModifierCompiler extends Base {
 			}
 			switch ($esc_type) {
 				case 'html':
+				case 'force':
+					// in case of auto-escaping, and without the 'force' option, no double-escaping
+					if ($compiler->getSmarty()->escape_html && $esc_type != 'force')
+						return $params[0];
+					// otherwise, escape the variable
 					return 'htmlspecialchars((string)' . $params[ 0 ] . ', ENT_QUOTES, ' . var_export($char_set, true) . ', ' .
 						var_export($double_encode, true) . ')';
 				// no break
 				case 'htmlall':
+					$compiler->setRawOutput(true);
 					return 'htmlentities(mb_convert_encoding((string)' . $params[ 0 ] . ', \'UTF-8\', ' .
 						var_export($char_set, true) . '), ENT_QUOTES, \'UTF-8\', ' .
 						var_export($double_encode, true) . ')';
 				// no break
 				case 'url':
+					$compiler->setRawOutput(true);
 					return 'rawurlencode((string)' . $params[ 0 ] . ')';
 				case 'urlpathinfo':
+					$compiler->setRawOutput(true);
 					return 'str_replace("%2F", "/", rawurlencode((string)' . $params[ 0 ] . '))';
 				case 'quotes':
+					$compiler->setRawOutput(true);
 					// escape unescaped single quotes
 					return 'preg_replace("%(?<!\\\\\\\\)\'%", "\\\'", (string)' . $params[ 0 ] . ')';
 				case 'javascript':
+					$compiler->setRawOutput(true);
 					// escape quotes and backslashes, newlines, etc.
 					// see https://html.spec.whatwg.org/multipage/scripting.html#restrictions-for-contents-of-script-elements
 					return 'strtr((string)' .
@@ -53,4 +63,4 @@ class EscapeModifierCompiler extends Base {
 		}
 		return '$_smarty_tpl->getSmarty()->getModifierCallback(\'escape\')(' . join(', ', $params) . ')';
 	}
-}
+}

src/Compile/Modifier/RawModifierCompiler.php

@@ -0,0 +1,21 @@
+<?php
+namespace Smarty\Compile\Modifier;
+
+use Smarty\Exception;
+
+/**
+ * Smarty raw modifier plugin
+ * Type:     modifier
+ * Name:     raw
+ * Purpose:  when escaping is enabled by default, generates a raw output of a variable
+ *
+ * @author Amaury Bouchard
+ */
+
+class RawModifierCompiler extends Base {
+
+	public function compile($params, \Smarty\Compiler\Template $compiler) {
+		$compiler->setRawOutput(true);
+		return ($params[0]);
+	}
+}

src/Compile/ModifierCompiler.php

@@ -75,7 +75,7 @@ class ModifierCompiler extends Base {
 				}
 			}
 		}
-		return $output;
+		return (string)$output;
 	}
 
 	/**

src/Compile/PrintExpressionCompiler.php

@@ -82,12 +82,13 @@ class PrintExpressionCompiler extends Base {
 					$output = $compiler->compileModifier($modifierlist, $output);
 				}
 
-				if ($compiler->getTemplate()->getSmarty()->escape_html) {
+				if ($compiler->getTemplate()->getSmarty()->escape_html && !$compiler->isRawOutput()) {
 					$output = "htmlspecialchars((string) ({$output}), ENT_QUOTES, '" . addslashes(\Smarty\Smarty::$_CHARSET) . "')";
 				}
 
 			}
 			$output = "<?php echo {$output};?>\n";
+			$compiler->setRawOutput(false);
 		}
 		return $output;
 	}

src/Compile/Tag/Section.php

@@ -99,7 +99,7 @@ class Section extends ForeachSection {
 
 		if ($compiler->tag_nocache) {
 			// push a {nocache} tag onto the stack to prevent caching of this block
-			$this->openTag('nocache');
+			$this->openTag($compiler, 'nocache');
 		}
 
 		$this->openTag($compiler, 'section', ['section', $compiler->tag_nocache]);

src/Compile/Tag/SectionClose.php

@@ -33,7 +33,7 @@ class SectionClose extends Base {
 
 		if ($nocache_pushed) {
 			// pop the pushed virtual nocache tag
-			$this->closeTag('nocache');
+			$this->closeTag($compiler, 'nocache');
 		}
 
 		$output = "<?php\n";

src/Compiler/Template.php

@@ -313,6 +313,12 @@ class Template extends BaseCompiler {
 	 */
 	private $noCacheStackDepth = 0;
 
+	/**
+	 * disabled auto-escape (when set to true, the next variable output is not auto-escaped)
+	 *
+	 * @var boolean
+	 */
+	private $raw_output = false;
 
 	/**
 	 * Initialize compiler
@@ -1486,4 +1492,21 @@ class Template extends BaseCompiler {
 	public function getTagStack(): array {
 		return $this->_tag_stack;
 	}
+
+	/**
+	 * Should the next variable output be raw (true) or auto-escaped (false)
+	 * @return bool
+	 */
+	public function isRawOutput(): bool {
+		return $this->raw_output;
+	}
+
+	/**
+	 * Should the next variable output be raw (true) or auto-escaped (false)
+	 * @param bool $raw_output
+	 * @return void
+	 */
+	public function setRawOutput(bool $raw_output): void {
+		$this->raw_output = $raw_output;
+	}
 }

src/Data.php

@@ -290,7 +290,7 @@ class Data
 	 * @return bool
 	 */
 	public function hasVariable($varName): bool {
-		return !($this->getVariable($varName) instanceof UndefinedVariable);
+		return !($this->getVariable($varName, true, false) instanceof UndefinedVariable);
 	}
 
 	/**

src/Extension/DefaultExtension.php

@@ -35,6 +35,7 @@ class DefaultExtension extends Base {
 			case 'lower': $this->modifiers[$modifier] = new \Smarty\Compile\Modifier\LowerModifierCompiler(); break;
 			case 'nl2br': $this->modifiers[$modifier] = new \Smarty\Compile\Modifier\Nl2brModifierCompiler(); break;
 			case 'noprint': $this->modifiers[$modifier] = new \Smarty\Compile\Modifier\NoPrintModifierCompiler(); break;
+			case 'raw': $this->modifiers[$modifier] = new \Smarty\Compile\Modifier\RawModifierCompiler(); break;
 			case 'round': $this->modifiers[$modifier] = new \Smarty\Compile\Modifier\RoundModifierCompiler(); break;
 			case 'str_repeat': $this->modifiers[$modifier] = new \Smarty\Compile\Modifier\StrRepeatModifierCompiler(); break;
 			case 'string_format': $this->modifiers[$modifier] = new \Smarty\Compile\Modifier\StringFormatModifierCompiler(); break;
@@ -753,4 +754,4 @@ class DefaultExtension extends Base {
 		return $string;
 	}
 
-}
+}

src/Resource/FilePlugin.php

@@ -56,11 +56,14 @@ class FilePlugin extends BasePlugin {
 	 * @param Source $source source object
 	 */
 	public function populateTimestamp(Source $source) {
-		if (!$source->exists && $path = $this->getFilePath($source->name, $source->getSmarty(), $source->isConfig)) {
-			$source->timestamp = $source->exists = is_file($path);
+		$path = $this->getFilePath($source->name, $source->getSmarty(), $source->isConfig);
+		if (!$source->exists) {
+			$source->exists = ($path !== false && is_file($path));
 		}
-		if ($source->exists && $path) {
+		if ($source->exists && $path !== false) {
 			$source->timestamp = filemtime($path);
+		} else {
+			$source->timestamp = 0;
 		}
 	}
 

src/Smarty.php

@@ -54,7 +54,7 @@ class Smarty extends \Smarty\TemplateBase {
 	/**
 	 * smarty version
 	 */
-	const SMARTY_VERSION = '5.2.0';
+	const SMARTY_VERSION = '5.4.0';
 
 	/**
 	 * define caching modes
@@ -2227,22 +2227,6 @@ class Smarty extends \Smarty\TemplateBase {
 	}
 
 	/**
-	 * Whether Smarty displays an error when using an unassigned variable
-	 */
-	public function getErrorUnassigned(): bool
-	{
-		return (bool) $this->error_unassigned;
-	}
-
-	/**
-	 * Set if Smarty should display an error on using an unassigned variable
-	 */
-	public function setErrorUnassigned(bool $error_unassigned): void
-	{
-		$this->error_unassigned = $error_unassigned;
-  }
-  
-  /**
 	 * Sets if Smarty should check If-Modified-Since headers to determine cache validity.
 	 * @param bool $cache_modified_check
 	 * @return void
@@ -2252,3 +2236,4 @@ class Smarty extends \Smarty\TemplateBase {
 	}
 
 }
+

src/Template/Compiled.php

@@ -136,7 +136,7 @@ class Compiled extends GeneratedPhpFile {
 		if ($this->exists && !$_smarty_tpl->getSmarty()->force_compile
 			&& !($_smarty_tpl->compile_check && $_smarty_tpl->getSource()->getTimeStamp() > $this->getTimeStamp())
 		) {
-			$this->loadCompiledTemplate($_smarty_tpl);
+			$this->loadCompiledTemplate($_smarty_tpl, false);
 		}
 
 		if (!$this->isValid) {
@@ -241,16 +241,19 @@ class Compiled extends GeneratedPhpFile {
 	 * HHVM requires a workaround because of a PHP incompatibility
 	 *
 	 * @param Template $_smarty_tpl do not change/remove variable name, is used by compiled template
+	 * @param bool $invalidateCachedFiles forces a revalidation of the file in opcache or apc cache (if available)
 	 *
 	 */
-	private function loadCompiledTemplate(Template $_smarty_tpl) {
-
-		if (function_exists('opcache_invalidate')
-			&& (!function_exists('ini_get') || strlen(ini_get("opcache.restrict_api")) < 1)
-		) {
-			opcache_invalidate($this->filepath, true);
-		} elseif (function_exists('apc_compile_file')) {
-			apc_compile_file($this->filepath);
+	private function loadCompiledTemplate(Template $_smarty_tpl, bool $invalidateCachedFiles = true) {
+        
+		if ($invalidateCachedFiles) {
+			if (function_exists('opcache_invalidate')
+				 && (!function_exists('ini_get') || strlen(ini_get("opcache.restrict_api")) < 1)
+			) {
+				opcache_invalidate($this->filepath, true);
+			} elseif (function_exists('apc_compile_file')) {
+				apc_compile_file($this->filepath);
+			}
 		}
 		if (defined('HHVM_VERSION')) {
 			eval('?>' . file_get_contents($this->filepath));

src/debug.tpl

@@ -144,6 +144,7 @@
                         {$vars['attributes']|debug_print_var nofilter}
                     {/if}
                 </td>
+            </tr>
          {/foreach}
     </table>
 

tests/UnitTests/A_Core/AutoEscape/AutoEscapeTest.php

@@ -61,4 +61,68 @@ class AutoEscapeTest extends PHPUnit_Smarty
         $this->assertEquals("<p>hi</p>", $this->smarty->fetch($tpl));
     }
 
+    /**
+     * test autoescape + raw modifier
+     */
+    public function testAutoEscapeRaw() {
+        $tpl = $this->smarty->createTemplate('eval:{$foo|raw}');
+        $tpl->assign('foo', '<a@b.c>');
+        $this->assertEquals("<a@b.c>", $this->smarty->fetch($tpl));
+    }
+
+    /**
+     * test autoescape + escape modifier = no double-escaping
+     */
+    public function testAutoEscapeNoDoubleEscape() {
+        $tpl = $this->smarty->createTemplate('eval:{$foo|escape}');
+        $tpl->assign('foo', '<a@b.c>');
+        $this->assertEquals("&lt;a@b.c&gt;", $this->smarty->fetch($tpl));
+    }
+
+    /**
+     * test autoescape + escape modifier = force double-escaping
+     */
+    public function testAutoEscapeForceDoubleEscape() {
+        $tpl = $this->smarty->createTemplate('eval:{$foo|escape:\'force\'}');
+        $tpl->assign('foo', '<a@b.c>');
+        $this->assertEquals("&amp;lt;a@b.c&amp;gt;", $this->smarty->fetch($tpl));
+    }
+
+    /**
+     * test autoescape + escape modifier = special escape
+     */
+    public function testAutoEscapeSpecialEscape() {
+        $tpl = $this->smarty->createTemplate('eval:{$foo|escape:\'url\'}');
+        $tpl->assign('foo', 'aa bb');
+        $this->assertEquals("aa%20bb", $this->smarty->fetch($tpl));
+    }
+
+    /**
+     * test autoescape + escape modifier = special escape
+     */
+    public function testAutoEscapeSpecialEscape2() {
+        $tpl = $this->smarty->createTemplate('eval:{$foo|escape:\'url\'}');
+        $tpl->assign('foo', '<BR>');
+        $this->assertEquals("%3CBR%3E", $this->smarty->fetch($tpl));
+    }
+
+    /**
+     * test autoescape + escape modifier = special escape
+     */
+    public function testAutoEscapeSpecialEscape3() {
+        $tpl = $this->smarty->createTemplate('eval:{$foo|escape:\'htmlall\'}');
+        $tpl->assign('foo', '<BR>');
+        $this->assertEquals("&lt;BR&gt;", $this->smarty->fetch($tpl));
+    }
+
+
+    /**
+     * test autoescape + escape modifier = special escape
+     */
+    public function testAutoEscapeSpecialEscape4() {
+        $tpl = $this->smarty->createTemplate('eval:{$foo|escape:\'javascript\'}');
+        $tpl->assign('foo', '<\'');
+        $this->assertEquals("<\\'", $this->smarty->fetch($tpl));
+    }
+
 }

tests/UnitTests/SmartyMethodsTests/GetTemplateVars/cache/.gitignore

@@ -1,2 +0,0 @@
-# Ignore anything in here, but keep this directory
-*

tests/UnitTests/SmartyMethodsTests/GetTemplateVars/templates_c/.gitignore

@@ -1,2 +0,0 @@
-# Ignore anything in here, but keep this directory
-*

tests/UnitTests/SmartyMethodsTests/TemplateVars/HasVariableTest.php

@@ -0,0 +1,32 @@
+<?php
+
+/**
+ * Tests the ::hasVariable method
+ */
+class HasVariableTest extends PHPUnit_Smarty
+{
+	public function setUp(): void
+	{
+		$this->setUpSmarty(__DIR__);
+	}
+
+
+	public function testInit()
+	{
+		$this->cleanDirs();
+	}
+
+	public function testSimpleTrue()
+	{
+		$this->smarty->assign('foo', 'bar');
+		$this->assertTrue($this->smarty->hasVariable('foo'));
+	}
+
+
+	public function testSimpleFalse()
+	{
+		$this->smarty->assign('foo', 'bar');
+		$this->assertFalse($this->smarty->hasVariable('foox'));
+	}
+
+}

tests/UnitTests/TemplateSource/TagTests/Section/CompileSectionTest.php

@@ -148,4 +148,10 @@ class CompileSectionTest extends PHPUnit_Smarty
         );
     }
 
+    public function testSectionWithNocache()
+    {
+        $source = 'string:{section name=module start=0 loop=1 nocache}{/section}';
+        $this->assertEquals('', $this->smarty->fetch($source));
+    }
+
 }